Building security and emergency detection and advisement system

ABSTRACT

Disclosed are systems and methods for providing distributed security event monitoring. The system can include a central monitoring system and sensor devices positioned throughout a premises that passively detect conditions and emit signals guiding people on the premises when a security event is detected. The sensor devices can include suites of sensors and can transmit detected conditions to the central monitoring system. The central monitoring system can combine the detected conditions to generate a collective of detected conditions, determine whether the collective of detected conditions exceeds expected threshold conditions, identify a security event on the premises based on the collective of detected conditions, classify the security event using machine learning models, generate instructions to produce audio or visual output at the sensor devices that notifies people on the premises about the security event, and transmit the instructions to the sensor devices to emit signals indicating information about the security event.

TECHNICAL FIELD

This document generally describes technology for detecting securityevents in a building from a variety of anomalous signals andautomatically determining a floor map of a building to use for advisingbuilding occupants how to exit the building during an emergency ordetected security event.

BACKGROUND

Buildings and homes can be equipped with security systems that identifysecurity events, such as theft and/or burglary. The security systems cancontinuously monitor the buildings and homes, and notify occupants whena security event is detected. Some security systems can send alerts tomobile devices of the occupants. Some security systems can also contactpolice or other law enforcement. Some security systems can also outputaudio signals, such as alarms, in the building that make the occupantsaware that a security event has been detected and also as a means toprovoke a burglar or other criminal to leave the premises withoutcausing more damage or harm.

For some types of emergencies and security events, such as fires,occupants can be urged to create emergency escape plans. The emergencyescape plans can be in written document form and can include maps of abuilding layout, immediate exterior grounds, and details on windows,doors, and other exits from the building. The occupants can also beurged to develop recommendations for escaping the building duringemergencies, which can take into account characteristics of eachoccupant as well as potential pathways that can be taken to exit fromeach room in the building. Fire and other emergency districts recommendthat occupants become familiar with their escape plans and to rehearsethem frequently. However, many occupants fail to create escape plans orupdate existing escape plans, let alone practice the plans regularly.Occupants may not be prepared in the event that a security event orother emergency is detected in the building.

SUMMARY

This document generally describes technology for detecting andidentifying security events using signals from a distributed group ofsensor devices within a premises (e.g., building, home, apartment,etc.). For example, a premises can be equipped with a group of sensordevices that are designed to unobtrusively monitor for conditions intheir proximate area (e.g., within a room where the sensor devices arelocated), such as through being embedded within a light switch, outletcover, light fixture, and/or other preexisting devices, structures,and/or features within a premises. Such sensor devices can include acollection of sensors that are configured to detect various conditions,such as microphones to detect sound, cameras to detect visual changes,light sensors to detect changes in lighting conditions, motion sensorsto detect nearby motion, temperature sensors to detect changes intemperature, accelerometers to detect movement of the devicesthemselves, and/or other sensors. Such sensor devices can additionallyinclude signaling components that are capable of outputting informationto people that are nearby, such as speakers, projectors, and/or lights.These signaling components can output information, such as informationidentifying safe pathways for emergency egress of a building.

Sensor devices can be positioned throughout a premises and can providesensed information about the environment within the premises to acentral computing device and/or system. The central computing deviceand/or system can combine and use the sensed information to collectivelydetect security related events (e.g., emergencies) within the premises(and to distinguish between security related events and non-securityevents). Such a central computing device and/or system can include, forexample, a device that is located within the premises and thatcommunicates with the sensor devices (e.g., wireless communication,wired communication). The device located within the premises can processthe signals locally and/or in combination with a remote computer system(e.g., cloud-based computing system). The central computing deviceand/or system can generate specific outputs that are appropriate fordetected events and conditions within the premises, such as transmittingalerts and/or messages regarding related events to user devicesassociated with the premises (e.g., home automation devices, mobiledevices, smart devices, etc.), causing one or more of the sensor devicesto collectively output egress and/or other guidance to people within thepremises, communicating with appropriate emergency personnel (e.g., firedepartment, police, emergency medical services), and/or othercommunication.

For example, in a house that includes sensor devices that are positionedin light switches and/or outlets throughout the house, when a windowbreaks in the house, each of the sensors devices may detect one or moresignals associated with the window breaking, such as audio from thesound of the window breaking, an increase or decrease in light within aroom from glass and/or other debris being scattered when the windowbreaks, and/or vibrations from the window breaking. This detectedinformation (e.g., conditions, intensity, duration) can be transmittedto the centralized computing device and/or system along with timinginformation (e.g., timestamp for when conditions are detected). Thecentralized computing device and/or system can use this informationalong with the relative timing of the detected conditions in combinationwith a physical relationship of the sensor devices to each other withinthe premises (e.g., a map of the positioning of the sensor deviceswithin the premises) to determine a type of event, a severity of theevent, a location of the event within the premises, and/or otherevent-related information.

For instance, the sensor devices that are located within a room wherethe window broke may detect the event with a greatest decibel level,with a greatest vibration level, and as occurring the earliest in timerelative to conditions detected by other sensor devices located in otherrooms (e.g., since sound travels slower than light and may dissipate asit is absorbed by walls, furniture, and other items in a house, sensordevices located in other rooms may detect the window breaking later intime and with a smaller decibel level). The centralized computing deviceand/or system can compare the received information and, using thatinformation, may accurately classify the location where the eventoccurred (e.g., triangulate the location based on relative intensity andtiming of signals detected by devices), the type and/or severity ofevent (e.g., categorize the type and/or severity based on patterns ofdetected conditions across different sensor devices), and othersecurity-related information. Having classified the event, its location,the potential danger posed (e.g., based on event type and severity),and/or other event information, an appropriate output can be generated,such as transmitting alerts to user devices associated with the premises(e.g., transmitting push notifications to user smartphones), outputtingappropriate guidance to people within the premises (e.g., directing themon safe pathway out of the premises), and/or alerting authorities.

To protect user privacy, the sensor devices may be limited to and/orrestricted from detecting and transmitting particular subsets ofinformation available for sensor-based detection. For example, amicrophone that is included as part of a sensor device may be restrictedto detecting decibel levels at one or more frequencies (and/or groups offrequencies) instead of detecting and transmitting entire audiowaveforms. Similarly, cameras and/or light sensors may be restricted todetecting intensities of and/or changes in light across one or morefrequencies and/or ranges of frequencies in the electromagneticspectrum, such as the visible spectrum, the infrared spectrum, and/orothers. Such sensor devices may be configured with such restrictions soas to permit for the devices to detect and/or transmit relevantinformation while at the same time avoiding potential issues related tocybersecurity that, if exploited, could provide an unauthorized thirdparty with access to private information regarding the premises. Byrestricting the functionality of sensors within the sensor devices(e.g., hardware-based restrictions, firmware-based restrictions),regardless of cybersecurity threats, the sensors are not be able todetect and/or transmit higher-fidelity information from within thepremises, while at the same time still being able to detect informationwith sufficient granularity to provide useful and actionable signals formaking security event determinations. However, in some instances, thesensors within the sensor devices may be unrestricted.

The sensor devices may further be configured to perform some localprocessing of signals to detect events before transmitting eventinformation to the centralized computer device/system. For example,instead of transmitting a stream of decibel information to a centralizedcomputer device/system for analysis to detect security related events,the sensor devices may perform local processing of the detected audioand, once one or more conditions and/or patterns have been detected,transmit audio information to the centralized computer device/systemdetailing that detected security related event. Such information caninclude, for example, the detected audio information and/or conditionsthat were satisfied to warrant the transmission to the centralizedcomputer device/system, as well as a timestamp indicating when the audiowas detected. The centralized computer device/system can synchronizeclocks for the sensor devices so that when events are detected by thesensor devices, timestamps can be attached to that information across anormalized timescale. Therefore, the centralized computer device/systemcan identify relative timing of detected events across the differentsensor devices.

The disclosed technology can determine a floor map of the premises touse for advising people within the premises about how to exit duringdetected security events or other emergencies. The centralized computingdevice/system can also create escape plans, based on pre-emergency andin-emergency motion detection of people within the premises and thefloor map. The centralized computing device/system can identify andguide people to essential egress routes, which can be critical if thepeople have not created or rehearsed escape plans in advance. Forexample, it can be challenging or even impossible for people to rehearseescaping a premises when a burglar breaks in. The centralized computingdevice/system can therefore generate escape plans in such scenarios andprovide instructions to the people that help them to safely and calmlyescape the premises at the time of the detected burglary.

As another example, during a fire or other emergency, strategicallyplaced voice and/or visual guidance prompts outputted by the sensordevices, user devices, or other devices positioned throughout thepremises can instruct people in each room of the premises regardingeffective egress paths, based on previously automatically identifiedfloor maps and current emergency location information. The techniquesdescribed herein can be used to detect and advise for a variety of typesof emergencies and security events, including but not limited to fires,gas leakage, water leaks, thieves entering the premises, burglary,carbon dioxide detection, and other types of security related events.

The disclosed techniques can also include an augmented reality (AR)interface to provide people with information about states of thepremises (e.g., real-time information) and potential security events andnon-security events. The sensor devices positioned throughout thepremises can, for example, recognize voices of people within thepremises. The sensor devices can learn which voices are associated withwhich people and can learn different tones and/or inflections in thevoices. At any time, the people can speak out loud in locations that areproximate to the sensor devices and ask for a current state of thepremises. The people can also ask for any other types of updates in thepremises (e.g., where a child is, what the child is doing, whetherthere's a break-in or other emergency, what a loud sound in the kitchenwas, etc.). The sensor devices can then perform the techniques describedherein to detect and/or the current state of the premises. As describedthroughout this disclosure, the sensor devices may also continuously andpassively monitor the premises to detect and determine the current stateof the premises. If the sensor devices do not detect a security relatedevent, one or more of the sensor devices can output audio (or some otherform of output) to the people that tells them everything is normalwithin the premises. The sensor devices can also provide output (e.g.,audio) that answers whatever question the people or an occupant/user maypose (such as asking a sensor device what the loud sound in the kitchenwas). If the sensor devices detect a security related event, the sensordevices can output a notification of the detected event and, based onthe tone or inflection in the peoples' voices, instructions or guidanceto help the people safely and calmly address the detected security event(e.g., egressing the premises along one or more predetermined escapeplans).

One or more embodiments described herein can include a system forproviding distributed security event monitoring in a premises. Thesystem can include a central monitoring system for detecting a securityevent in a premises and a plurality of sensor devices positionedthroughout the premises. The plurality of sensor devices can beconfigured to (i) passively detect conditions on the premises and (ii)emit signals that indicate guidance for people in the premises when thesecurity event is detected by the central monitoring system. Each of theplurality of sensor devices can include a suite of sensors. Each of theplurality of sensor devices can be configured to passively detect, usingthe suite of sensors, the conditions on the premises, and transmit thedetected conditions to the central monitoring system. The centralmonitoring system can be configured to receive the detected conditionsfrom one or more of the plurality of sensor devices, combine thedetected conditions to generate a collective of detected conditions,determine whether the collective of detected conditions exceeds expectedthreshold conditions for the premises beyond a threshold amount, andidentify, based on determining that the collective of detectedconditions exceeds the expected threshold conditions beyond thethreshold amount, a security event on the premises. The centralmonitoring system can also classify the security event using one or moremachine learning models that were trained to identify a type of thesecurity event using training data that correlates information aboutconditions detected on premises with different types of security events,and generate, based on the classified security event, instructions toproduce audio or visual output at the plurality of sensor devices. Theoutput can notify the people on the premises about the security event.The central monitoring system can also transmit the instructions to oneor more of the plurality of sensor devices for the one or more of theplurality of sensor devices to emit signals indicating information aboutthe security event.

The one or more embodiments can optionally include one or more of thefollowing features. For example, the type of the security event caninclude at least one of a burglary, a theft, a break-in, a fire, a gasleak, and a flood. Classifying the security event using one or moremachine learning models can include determining, based on (i) the typeof the security event and (ii) a magnitude in deviation of thecollective of detected conditions from the expected thresholdconditions, a severity level of the security event, and determining alocation of the security event based on (i) a map of the premises thatwas generated by the central monitoring system, (ii) timestampsindicating when the conditions were detected by the plurality of sensordevices, and (ii) positioning information of each of the plurality ofsensor devices on the premises. Moreover, the central monitoring systemcan further be configured to transmit, based on determining that theseverity level of the security event exceeds a threshold reportinglevel, a notification about the security event to emergency responsepersonnel.

As another example, the expected threshold conditions can be normalconditions on the premises that have been identified by the centralmonitoring system based on a historic spread of conditions detected bythe plurality of sensor devices over a predetermined period of time.

Sometimes, the central monitoring system can also be configured togenerate instructions to produce (i) audio output when the collective ofdetected conditions satisfy a first output threshold condition and (ii)visual output when the collective of detected conditions satisfy asecond output threshold condition. The first output threshold conditioncan include detection of at least one of a fire, smoke, and anobstruction of visual output devices of the plurality of sensor devices.The second output threshold condition can include detection of at leastone of a fire, a break-in, a burglary, and an obstruction of audiooutput devices of the plurality of sensor devices.

The central monitoring system can also be further configured todetermine, before detection of a security event, a map of the premises,the map indicating routes within the premises, including exits out ofthe premises and determine, based on the map and the collective ofdetected conditions, one or more exit routes that can be used by peopleto exit the premises. The collective of detected conditions can includelocation information indicating one or more locations on the premiseswhere the security event may be located, and the exit routes can avoidthe one or more locations where the security event may be located. Thecentral monitoring system can also transmit signaling instructions toone or more of the plurality of sensor devices for the one or more ofthe plurality of sensor devices to emit signals that indicate to thepeople an exit route to exit the premises.

Sometimes, the emitted signals can comprise voice commands. The voicecommands can comprise at least one of instructions to guide the peopleto exit the premises and information about the security event. Theinformation about the security event can include the type of thesecurity event, a location of the security event, and a severity levelof the security event. Sometimes, the emitted signals can comprise lightsignals. The light signals can include directional signals that directthe people to an exit route that avoids a location of the securityevent.

As another example, the suite of sensors can include at least one of alight sensor, an audio sensor, a temperature sensor, a motion sensor, auser presence sensor, an image sensor, and a smoke sensor. The audiosensor can, for example, be configured to detect changes in decibels inan area proximate to a location of the audio sensor. Sometimes, theplurality of sensor devices can be integrated into at least one ofoutlet covers, light switches, alert systems, thermostats, and lightfixtures that are installed on the premises. One of the plurality ofsensor devices can sometimes be configured to operate as the centralmonitoring system and the other sensor devices of the plurality ofsensor devices can communicate amongst each other.

As yet another example, one or more of the plurality of sensor devicescan be configured to identify the security event and transmitinformation about the identified security event to the centralmonitoring system. The central monitoring system can then receive theidentified security event from one or more of the plurality of sensordevices, classify the security event, generate instructions to produceoutput by the plurality of sensor devices about the security event, andtransmit the instructions to the plurality of sensor devices.

One or more of the plurality of sensor devices can also be configured todetect audio input from a person on the premises and transmit the audioinput and a respective timestamp to the central monitoring system. Theaudio input can include a request for information about a current stateof the premises. The central monitoring system can then be configured toreceive the audio input and the respective timestamp, transmit, based onthe request for information in the audio input, requests to each of theplurality of sensor devices for conditions detected at a similar time asthe timestamp, receive, from one or more of the plurality of sensordevices, the conditions detected at the similar time as the timestamp,identify the current state of the premises based on comparing thedetected conditions to historic threshold conditions for the premises atthe similar time as the timestamp, generate instructions for the one ormore of the plurality of sensor devices to provide audio output to theperson indicating the current state of the premises, and transmit, tothe one or more of the plurality of sensor devices, the instructions toprovide audio output to the person.

As yet another example, the central monitoring system can further beconfigured to transmit the instructions to provide output to one or moremobile devices of the people on the premises. The instructions can causethe one or more mobile devices to output at least one of audio signals,text messages, and push notifications about the security event.Moreover, each of the plurality of sensor devices further can include anaudio signal generator and a visual signal generator. The visual signalgenerator can include a projector that projects a lighted sign on asurface, and the surface can be one or more of a wall, a floor, and aceiling on the premises. The premises can be at least one of a building,a home, and an apartment.

One or more embodiments described herein can also include a method forproviding distributed security event monitoring in a premises. Themethod can include receiving, by a computing system and from a pluralityof sensor devices, detected conditions on a premises. The plurality ofsensor devices can include a suite of sensors that passively detect theconditions on the premises, and the plurality of sensor devices can bepositioned throughout the premises. The method can also includecombining, by the computing system, the detected conditions to generatea collective of detected conditions, determining, by the computingsystem, whether the collective of detected conditions exceeds expectedthreshold conditions for the premises beyond a threshold amount,identifying, by the computing system and based on determining that thecollective of detected conditions exceeds the expected thresholdconditions beyond the threshold amount, a security event on thepremises, classifying, by the computing system, the security event usingone or more machine learning models that were trained to identify a typeof the security event using training data that correlates informationabout conditions detected on premises with different types of securityevents, generating, by the computing system and based on the classifiedsecurity event, instructions to produce audio or visual output at theplurality of sensor devices, and transmitting, by the computing systemto one or more of the plurality of sensor devices, the instructions forthe one or more of the plurality of sensor devices to emit signalsindicating information about the security event. The output can notifythe people on the premises about the security event.

The method can optionally include one or more of the abovementionedfeatures. As another example, the computing system can be at least oneof remote from the premises, centralized at the premises, and one of theplurality of sensor devices.

The details of one or more implementations are depicted in theassociated drawings and the description thereof below. Certainimplementations of the disclosed techniques may provide one or moreadvantages. For example, emergency plans can be generated, and used indetected security events, even if people within the premises have notpreviously generated, reviewed, practiced, or seen emergency plans.Dynamic evacuation guidance can be provided that is based on real-timesituational information about people and compromised location(s) withinthe premises. Real-time information about a current detected securityevent can be exchanged between sensor devices positioned within thepremises, mobile devices or other user devices of the people on thepremises, a centralized computer device/system, and devices/systems ofemergency response personnel/teams. This communication can be beneficialto ensure that people within the premises are made aware of securityrelated events as soon as they happen and respond safely and calmly. Thedisclosed technology can therefore provide people with appropriateguidance and notifications that help the people to reach safety,regardless of the type of security event that is detected on thepremises.

Moreover, the centralized computer device/system can evaluate possibleescape routes, determined from automatically generated floor maps,select recommended escape route(s), and instruct sensor devicespositioned throughout the premises and/or user devices of the peoplewithin the premises to inform the people of the recommended escaperoute(s). As a result, when a security event is detected, thecentralized computer device/system may determine how the people canreach safety and guidance to get the people quickly and calmly tosafety.

As another example, the disclosed techniques provide for passivelymonitoring conditions in the premises to detect security related eventswhile also preserving privacy of people on the premises. Passivemonitoring can include collection of anomalous signals, such as changesin lighting, temperature, motion, and/or decibel levels and comparingthose anomalous signals to normal conditions for the premises. Suddenchanges in decibel levels and/or temperature levels, for example, can beindicative of a security related event. The sensor devices, therefore,may be restricted to detect particular types of signals that do notinvolve higher-fidelity information from within the premises but enoughgranularity to provide useful and actionable signals for making securityevent determinations. Thus, the people on the premises may not betracked as they go about their daily lives and their privacy can bepreserved.

The disclosed techniques can also provide for accurately identifyingtypes of security events, locations of such security events, andseverity of the security events. The centralized computer device/systemcan, for example, be trained to identify different types of securityevents by correlating various different signals captured during asimilar timeframe and from sensor devices in different locationthroughout the premises. The centralized computer device/system can, forexample, correlate audio signals indicating a sharp increase in soundlike a glass window breaking, motion signals indicating sudden movementnear a window where the audio signals were captured, and a video feed orother image data showing a body moving near the window where the audiosignals were captured to determine where and when a break-in occurred onthe premises. The centralized computer device/system can also be trainedto determine a severity of this identified security event based on thelinked signals. Using the determined information, the centralizedcomputer device/system can also identify appropriate action(s) to take,such as notifying people on the premises, notifying emergency responsepersonnel, and/or providing the people with guidance to safely, quickly,and calmly egress using one or more predetermined escape plans.

The disclosed techniques also provide for seamless and unobtrusiveintegration of sensor devices and existing features on the premises.Sensor devices described herein can be integrated into wall outlets,lightbulbs, fans, light switches, and other features that may alreadyexist on the premises. Existing sensor devices, such as fire alarms,smoke detectors, temperature sensors, motion sensors, and/or existingsecurity systems can also be retrofitted in the building to communicatedetected signals with installed sensor devices, user devices, mobiledevices of people on the premises, and the centralized computerdevice/system described herein. Such seamless and unobtrusiveintegration can provide for performing the techniques describedthroughout this disclosure without interfering with normal activities ofpeople on the premises and an aesthetic design/appeal of the premises.

Additionally, the disclosed techniques provide for robust securitysystems that incorporate artificial intelligence (AI) and/or augmentedreality (AR). Seamless integration of sensor devices on the premises canprovide more intuitive and unobtrusive detection of security relatedevents on the premises. Such seamless integration can also make iteasier for people on the premises to learn about and get updates aboutcurrent activity on the premises. For example, if an occupant hears anabnormal sound, such as a crashing sound, the occupant can speak outloud by asking a sensor device located in a same room as the occupantwhether everything is okay. The sensor device can ping other sensordevices on the premises for sensed conditions and can determine, basedon those sensed conditions, whether a security related event has beendetected. Based on this determination, the sensor device can respond tothe occupant and output information (e.g., audio output) that informsthe occupant about the current state of the premises. This provides amore intuitive and easier-to-use user interface for the people on thepremises, especially if the people are concerned, nervous, or otherwisein a state of frenzy. The people can therefore receive updates aboutconditions on the premises and/or request updates on conditions on thepremises without having to use additional devices such as mobile devicesor mobile applications.

As yet another example, the disclosed techniques provide for localprocessing at sensor devices to avoid clogging network bandwidth. Sincesome processing, such as determining whether detected signals areindicative of a security related event, can be performed locally at asensor device, computing resources can be better allocated andefficiently used at the centralized computer device/system. As a result,the centralized computer device/system can more quickly and efficientlyutilize available computer resources and network bandwidth to performother processing, including but not limited to detecting securityrelated events, assessing type and/or severity of such events,generating output and guidance for people on the premises, anddetermining appropriate egress plans on the premises.

Other features, objects, and advantages of the technology described inthis document will be apparent from the description and the drawings,and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a conceptual diagram of an example centralized hub fordetecting and identifying a security event in a building using a varietyof anomalous signals.

FIGS. 2A-C depict example system components that can be used to performthe techniques described herein.

FIG. 3 is a flowchart of a process for detecting a security event in abuilding.

FIGS. 4A-B is a flowchart of an example process for detecting a securityevent from irregular audio signals.

FIG. 5A is a conceptual diagram of an example scenario where an occupantverbally requests an update on conditions in a building from a sensordevice.

FIG. 5B is a conceptual diagram of another example scenario where theoccupant verbally requests an update on conditions in the building fromthe sensor device.

FIG. 6 depicts example features in a building that include sensordevices integrated therein.

FIG. 7 depicts example forms of output that can be generated based on adetected security event.

FIGS. 8A-B depict exemplary systems for providing emergency or securityevent detection, guidance and advisement.

FIG. 9 is an example apparatus for providing emergency guidance andadvisement.

FIG. 10 is another example apparatus for providing emergency guidanceand advisement.

FIG. 11 is a conceptual diagram depicting a building with sensor devicespositioned throughout that provide guidance and prompts to buildingoccupants during an identified emergency.

FIG. 12 is a schematic diagram that shows an example of a computingdevice and a mobile computing device.

Like reference symbols in the various drawings indicate like elements.

DETAILED DESCRIPTION

This document generally describes systems and methods for detectingsecurity events from anomalous signals and generating floor maps andguidance based on the floor maps to assist people on a premises, such asoccupants in a building, how to respond to the detected securityevent(s). For example, the building can be equipped with agroup/plurality of sensor devices that are designed to passively andunobtrusively monitoring conditions in the devices' proximate areas,such as a room where a sensor device is located. The sensor devices canbe embedded in existing features or fixtures in the building, such as alight switch, outlet cover, light fixture, and/or other preexistingdevices, structures, and/or features. The detected conditions can bereceived by a centralized hub (e.g., centralized computerdevice/system), which can determine whether the detected conditionsexceed expected threshold levels at a particular time when theconditions are detected and/or whether the detected conditions exceednormal conditions for the building that are learned over time. Thecentralized hub can use the detected conditions along with relativetiming of the detected conditions and a physical relationship of thesensor devices to each other in the building to determine and classify atype of security event, a severity of the event, a location of theevent, and/or other event related information. Other event relatedinformation can include instructions for guiding occupants of thebuilding to safety along one or more predetermined exit routes out ofthe building.

Referring to the figures, FIG. 1 is a conceptual diagram of an examplecentralized hub 102 for detecting and identifying a security event in abuilding 100 using a variety of anomalous signals. A variety ofdifferent types of security events can be detected using the disclosedtechniques. The security events can include, but are not limited to,break-ins, burglary, theft, natural disasters, fire, carbon monoxide,flood, gas leaks, and other types of emergencies. The centralized hub102 can be in communication via network(s) 114 with sensor devices108A-N and sensors 112A-N. The centralized hub 102, sensor devices108A-N, and/or sensors 112A-N can also be in communication vianetwork(s) 114 with a security system for the building 100.Communication can be wired and/or wireless (e.g., BLUETOOTH, WIFI,ETHERNET, etc.). Communication can also be through a home network.

The centralized hub 102 can be a computer system or network ofcomputers. The centralized hub 102 can be located at the building 100.For example, the centralized hub 102 can be one of the sensor devices108A-N. The centralized hub 102 can also be any other type of computingdevice and/or computer system that can be integrated into the building100. For example, the centralized hub 102 can be a computing device orother type of device with a display screen. The centralized hub 102 canbe located in a central location in the building 100, such as near afront door 110, and can provide building occupants with informationabout conditions in the building 100 as well as predetermined escapeplans. The building occupants can provide updated information about theoccupants and/or the building 100 to the centralized hub 102 via thedisplay screen. This input can then be used by the centralized hub 102to improve or otherwise modify security event detections and escape plandeterminations. The centralized hub 102 can also be remote from thebuilding 100. For example, the centralized hub 102 can be a cloud basedcomputing system that is configured to identify security events based onanomalous signals that are detected and received from sensor devices andsensors in different buildings.

The sensor devices 108A-N can be configured to continuously andpassively monitor conditions throughout the building 100. The sensordevices 108A-N can be networked with each other (e.g., via a homenetwork) such that they can communicate with each other. For example, ina scenario where the centralized hub 102 goes down or a connectionbetween the sensor devices 108A-N and the centralized hub 102 goes down,the sensor devices 108A-N can communicate with each other to determine astate of activity in the building 100 and whether a security event isdetected. Each of the sensor devices 108A-N can also operate as acentralized hub. Sometimes, each of the sensor devices 108A-N can taketurns operating as the centralized hub. Sometimes, one of the sensordevices 108A-N can be assigned as the centralized hub. When a sensordevice operates as the centralized hub, the senor device can ping orotherwise communicate with the other sensor devices to determine whenabnormal signals are detected in the building, whether a security eventhas or is occurring, and what guidance or instructions can be generatedand provided to building occupants.

As described further below, the sensor devices 108A-N can include asuite of sensors that passively monitor different conditions or signalsin the building 100. For example, the sensor devices 108A-N can includeaudio, light, visual, temperature, smoke, and/or motion sensors. Thesensors can pick up on or otherwise detect anomalous and random signals,such as changes in decibels, flashes of light, increases in temperature,strange odors, and/or sudden movements. Therefore, the sensor devices108A-N may not be actively monitoring building occupants as they goabout with their daily activities. In other words, to protect occupantprivacy, the sensor devices 108A-N can be limited to and/or restrictedto detecting intensities of and/or changes in different types ofconditions in the building 100. The sensor devices 108A-N may transmitparticular subsets of detected information so as to protect againstthird party exploitation of private information regarding the occupantsand the building 100. The sensor devices 108A-N can there detect and/ortransmit information such as changes or deltas in decibel levels, light,motion, movement, temperature, etc.

Some sensor devices 108A-N can include a first subset of sensors whileother sensor devices 108A-N can include a second, third, etc. subset ofsensors. As an example, a first sensor device located in a room caninclude audio and light sensors and a second sensor device located inthe same room can include image and temperature sensors. One or moreother suites of sensors in the sensor devices 108A-N positionedthroughout the building 100 are possible. The sensor devices 108A-N canalso include output devices, such as audio and visual outputs (e.g.,microphones, lights, speakers, projectors, etc.) so that the sensordevices 108A-N can provide guidance and/or instructions to buildingoccupants when a security event is detected. As described furtherherein, the sensor devices 108A-N can also provide output to thebuilding occupants that indicates a current state of the building 100.

For example, a building occupant can speak to one of the sensor devices108A-N and ask for an update on the state of the building 100. Using thetechniques described herein, the sensor device and/or the centralizedhub can determine the current state of the building 100. The sensordevice can generate audio output that tells the occupant what ishappening in the building 100. The sensor device can also generateoutput that visually represents the current state of the building 100.For example, using augmented reality (AR), the sensor device can projectvisual representations of the building 100 on a surface proximate to theoccupant (e.g., on a wall and/or a floor) that indicate the currentstate of the building 100. Visual representations can include an imageor video clip of a particular area in the building 100 where activitywas detected, text describing or explaining what activity (orinactivity) was detected in the building 100, and the like.

The sensor devices 108A-N can be unobtrusively integrated into thebuilding 100. Sometimes, the sensor devices 108A-N can be integrated orretrofitted into existing features in the building 100, such as in lightfixtures, light bulbs, light switches, power outlets, and/or outletcovers. The sensor devices 108A-N can also be standalone devices thatcan be installed in various locations throughout the building 100 so asto not interfere with the daily activities of the building occupants andto be relatively hidden from sight to preserve aesthetic appeal in thebuilding 100. For example, the sensor devices 108A-N can be installed incorners, along ceilings, against walls, etc.

In the example building 100, the sensor devices 108A-N are positioned ineach room 106A-C. Sensor devices 108B and 108N have been positioned inroom 106A. Sensor devices 108A and 108D have been positioned in room106B. Sensor device 108C has been positioned in room 106C. Multiplesensor devices can be positioned in one room. Sometimes, only one sensordevice can be positioned in a room. Moreover, sometimes a sensor devicemay not be positioned in a room. Instead, the room can include one ormore standalone sensors 112A-N.

The sensors 112A-N can be positioned throughout the building 100.Sometimes, the sensors 112A-N can already be installed in the building100 before the sensor devices 108A-N and the centralized hub 102 areadded to the building 100. For example, the sensors 112A-N can includeexisting security cameras, smoke detectors, fire alarms, user motionsensors, light sensors, etc. Some of the rooms 106A-C in the building100 can include the sensors 112A-N while other rooms may not. In someimplementations, one or more of the sensors 112A-N can be integratedinto or otherwise part of the sensor devices 108A-N. In the examplebuilding 100, sensor 112A is positioned in the room 106C and sensor 112Nis positioned in the room 106A. One or more additional sensors 112A-Ncan be installed throughout the building 100.

Still referring to FIG. 1 , the centralized hub 102 can synchronizeclocks of the sensor devices 108A-N and optionally the sensors 112A-N.Synchronization can occur at predetermined times, such as once a day,once every couple hours, at night, and/or during inactive times in thebuilding. To synchronize clocks, the centralized hub 102 can send asignal with a timestamp of an event to all of the sensor devices 108A-Nand optionally the sensors 112A-N. All the sensor devices 108A-N canthen synchronize their clocks to the timestamp such that they are on asame schedule. The centralized hub 102 can also transmit a signal to thesensor devices 108A-N and optionally the sensors 112A-N that indicates alocal clock time. The sensor devices 108A-N can then set their clocks tothe same local time. Synchronization makes matching up or linking ofdetected signals easier and more accurate to detect security events inthe building 100. In other words, when security events are detected bythe sensor devices 108A-N and/or the centralized hub 102, timestamps canbe attached to detected conditions/information across a normalizedscale. The centralized hub 102 and/or the sensor devices 108A-N canaccordingly identify relative timing of detected events across thedifferent sensor devices 108A-N, regardless of where the devices 108A-Nare located in the building 100.

At time=1, each of the sensor devices 108A-N and the sensors 112A-N canpassively monitor conditions in the building 100 (step A). For example,the sensor devices 108A-N can collect temperature readings in each ofthe rooms 106A-C during one or more times (e.g., every 2 minutes, every5 minutes, every 10 minutes, etc.). As another example, the sensordevices 108A-N can collect decibel readings in each of the rooms 106A-Cat predetermined times and/or continuously. The sensor devices 108A-Ncan collect one or more other signal readings as described herein (e.g.,motion, light, etc.).

As mentioned throughout, the sensor devices 108A-N can passively monitorconditions in such a way that protects occupant privacy. The sensordevices 108A-N may be limited to and/or restricted from detecting andtransmitting particular subsets of information available forsensor-based detection. For example, an audio sensor can be restrictedto detect only decibel levels at one or more frequencies (and/or groupsof frequencies) instead of detecting and transmitting entire audiowaveforms. Similarly, cameras, image sensors, and/or light sensors maybe restricted to detecting intensities of and/or changes in light acrossone or more frequencies and/or ranges of frequencies in theelectromagnetic spectrum, such as the visible spectrum, the infraredspectrum, and/or others. Configuring the sensor devices 108A-N with suchrestrictions allows for the devices 108A-N to detect and/or transmitrelevant information while at the same time avoiding potential issuesrelated to cybersecurity that, if exploited, could provide anunauthorized third party with access to private information regardingbuilding occupants. Although functionality of sensors within the sensordevices 108A-N may be restricted, the sensor devices 108A-N can stilldetect information with sufficient granularity to provide useful andactionable signals for making security event determinations and furtherclassifications.

One or more of the sensor devices 108A-N can detect an event at time=1(step B). In the example of FIG. 1 , the sensor device 108A detects anevent in the room 106B. The event can be a break-in, which isrepresented by a brick 104 being thrown through the door 110 and intothe room 106B. The sensor device 108A can detect the event in a varietyof ways. For example, at time=1, the sensor device 108A is passivelymonitoring signals (step A) in the room 106B, at which point a sharpincrease in decibel readings can be detected. The sharp increase indecibel readings can indicate the brick 104 being thrown through thedoor 110 and breaking glass of the door 110. The sharp increase indecibel readings may not be a normal condition or expected signal forthe building 100 or the particular room 106B. Thus, the sensor device108A can detect that some abnormal event has occurred at time=1 (stepB).

Likewise, the sensor device 108A can detect a sudden movement by thedoor 110, which can represent the brick 104 hitting the door 110 andlanding inside of the room 106B. The sudden detected movement may not bea normal condition or expected signal for the building 100, theparticular room 106B, and/or at time=1. Thus, the sensor device 108A candetect that some abnormal event has occurred at time=1 (step B).

The sensor device 108A can transmit the detected signal(s) at time=1 tothe centralized hub 102 (step C). Moreover, once an event is detected(step B), each of the sensor devices 108A-N and optionally the sensors112A-N can transmit any signals that were detected at time=1 to thecentralized hub 102. This is why synchronization of clocks of the sensordevices 108A-N and optionally the sensors 112A-N is performed. The othersensor devices 108B-N and the sensors 112A-N can be notified or pingedby the sensor device 108A when the event is detected (step B) so thatthe other sensor devices 108B-N and the sensors 112A-N can transmitsignals that they detected at the same time as the event (step C). Thedetected signal(s) can be transmitted to the centralized hub 102 withtiming information, such as a timestamp of when the signals and/orcondition were detected.

Sometimes, the sensor devices 108A-N and the sensors 112A-N cancontinuously transmit detected signals (step C) to the centralized hub102. Therefore, the centralized hub 102 can receive a string of signalsfrom one or more or all of the sensor devices 108A-N and the sensors112A-N. The sensor devices 108A-N and the sensors 112A-N can transmitdetected signals to the centralized hub 102 at predetermined times(e.g., every 1 minute, 2 minutes, 5 minutes, etc.). Sometimes, thesensor devices 108A-N and the sensors 112A-N can transmit detectedsignals based on receiving a request from the centralized hub 102 forthe signals. For example, the centralized hub 102 can ping the sensordevices 108A-N and the sensors 112A-N at predetermined times and requestthe sensor devices 108A-N and the sensors 112A-N for any changes indetected signals. If changes have been detected, then the signals can betransmitted to the centralized hub 102. If changes have not beendetected then the signals may not be transmitted to the centralized hub102. Moreover, as described above, the sensor devices 108A-N and thesensors 112A-N can be triggered to transmit signals to the centralizedhub only upon detection of some event by one or more of the sensordevices 108A-N and/or the sensors 112A-N (step B).

Once the centralized hub 102 receives the detected signals, thecentralized hub 102 can determine whether any of the signals exceedexpected threshold values (step D). The centralized hub 102 can combinethe detected signals into a collective of signals and determine whetherthe collective of signals exceeds expected threshold values for thebuilding 100. The centralized hub 102 can also use the relative timinginformation and physical relationship of the sensor devices 108A-N inthe building 100 to determine, for the signals or collective of detectedsignals that exceed the expected threshold values, a type of securityevent, a severity of the event, a location of the event, and/or otherevent-related information.

For example, as described further below, the centralized hub 102 cancompare the detected signals to historic signals that correspond to thebuilding 100 and/or the room in which the signal was detected. For thesharp increase in decibel readings in the room 106B, the centralized hub102 can compare this increase to expected decibel readings for the room106B. The expected decibel readings for the room 106B can be based onprevious decibel readings for the room at the same or similar time astime=1. For example, if time=1 is at 8:30 in the morning, the expecteddecibel readings can be a historic spread of decibel readings that weretaken at 8:30 in the morning over a certain number of days. At 8:30 inthe morning, historic changes in decibel readings can be very lowbecause building occupants may still be asleep at that time. Therefore,if the detected signals at time=1 is a sudden increase in decibelreadings that deviates from the expected signals at time=1, thecentralized hub 102 can determine that the detected signal likelyrepresents some type of security event.

The centralized hub 102 can also determine whether any of the otherreceived signals from the sensor devices 108A-N and the sensors 112A-Ndeviate from expected threshold values (step D). For example, the sensordevice 108N in the room 106A at a back of the building 100 can detect achange in decibel readings that, although may be less of a sharpincrease than the change in decibel readings detected by the sensordevice 108A, is abnormal for that room 106A and/or time. After all,sound can travel slower and can bounce off of objects or other featuresin the building 100, thereby causing the decibel readings in a differentarea of the building 100 to be of lesser magnitude. Since the clocks ofthe sensor devices 108A-N are synchronized and the sensor device 108Ndetected a change in decibel readings at the same or similar time thatthe sensor device 108A detected a change in decibel readings, thecentralized hub 102 can confirm that a security event was detected bythe sensor device 108A.

Accordingly, the centralized hub 102 can correlate the detected signals(step E). Correlating the signals can include linking signals thatdeviate from the expected threshold values to piece together andidentify the security event. As described herein, the centralized hub102 can correlate different types of signals to piece together thesecurity event. For example, the centralized hub 102 can link togetherdecibel signals from the sensor device 108A with motion signals from thesensor device 108D and decibel signals from one or more of the othersensor devices 108B, 108C, and 108N and/or the sensors 112A and 112N.The centralized hub 102 can also link together the above mentionedsignals with video or other image data that is captured by an existingsecurity system inside or outside the building 100. By correlatingdifferent types of detected signals, the centralized hub 102 can moreaccurately determine what type of security event has occurred at time=1.

The centralized hub 102 can identify the security event and location ofthe security event (step F). For example, the centralized hub 102 canuse one or more machine learning models that are trained to identify atype of security event from different types of signals, correlatedsignals, changes/deviations in signals, etc. The centralized hub 102 cancategorize the type of the security event based on patterns of detectedsignals across different sensor devices 108A-N and/or sensors 112A-N.The models can be trained using deep learning (DL) neural networks,convolutional neural networks (CNNs), and/or one or more other types ofmachine learning techniques, methods, and/or algorithms. The models canalso be trained using training data that includes signals that have beendetected by the sensor devices 108A-N and/or the sensors 112A-N in thebuilding 100. The models can be trained to identify security eventsbased on detected signals and expected conditions of the particularbuilding 100. The models can also be trained to identify security eventsbased on signals and expected conditions in a variety of differentbuildings.

The centralized hub 102 can determine a location of the security event(step F) based on where, for example, a deviation in detected signals isgreatest. The centralized hub 102 can also determine the location basedon video or other image data that can be received from one or more ofthe sensor devices 108A-N, the sensors 112A-N, and/or camera securitysystems installed at the building 100. The video or other image data cancapture the detected security event at time=1. In some implementations,the centralized hub 102 can determine the location by triangulating thelocation based on relative intensity and timing of signals that aredetected by the sensor devices 108A-N and/or the sensors 112A-N.

As described further below, the centralized hub 102 can also determine aseverity of the security event (step F). This determination can be madeusing one or more machine learning models. This determination can alsobe made based on comparing and correlating patterns of detected signalsacross the different sensor devices 108A-N and/or the sensors 112A-N.

Based on the identification of the security event, the location of thesecurity event, and the severity of the security event, the centralizedhub 102 can output an indication of the security event (step G). In someimplementations, the centralized hub 102 can generate instructions thatcan be used by the sensor devices 108A-N and/or one or more otherdevices in the building 100 to output information about the securityevent to building occupants.

As described further below, the centralized hub 102 can determine thatbuilding occupants should receive guidance to safely, calmly, andquickly exit the building 100. The centralized hub 102 can determinethat this guidance should be outputted as text messages to theoccupant's mobile devices instead of audio or visual outputs by thesensor devices 108A-N and/or the occupant's mobile devices. Thecentralized hub 102 can determine that audio or visual outputs mayincrease safety risks to the occupants since it can bring more attentionto them. The centralized hub 102 can also automatically contactemergency response personnel to report the break-in. Determining whetherto contact emergency response personnel can be based on the type ofsecurity event and the severity of the security event. When thecentralized hub 102 provides guidance to help the occupants safelyegress from the building 100, the centralized hub 102 can determineoptimal egress pathways based on a floorplan of the building 100,locations of the occupants in the building, and/or the location of thesecurity event, as described further below.

Although FIG. 1 is described in reference to an example break-insecurity event, the techniques described in FIG. 1 are not so limitingand apply to various types of security events and other emergencies. Forexample, steps A-G can be performed when fires, floods, gas leaks, andnatural disasters are detected. Steps A-G can also be performed whenother types of security-related events are detected in the building 100.The techniques described herein therefore provide a robust security andemergency detection and guidance system.

Moreover, in some implementations, one or more of the steps A-G can beperformed by one or more of the sensor devices 108A-N instead of at thecentralized hub 102. For example, instead of transmitting a stream ofdecibel information or other detected signals to the centralized hub 102for analysis to detect security related events, the sensor devices108A-N may perform local processing of the detected signals and identifypatterns amongst the detected signals (such as in steps D and E). Thesensor devices 108A-N can sometimes identify a security event andinformation about the security event (such as in step F), and thentransmit such information to the centralized hub 102. The centralizedhub 102 can perform additional processing, such as classifying thesecurity event based on type, location, and/or severity (such as in stepF). The centralized hub 102 can also determine what type of outputshould be generated and transmitted back to the sensor devices 108A-N orother devices in the building 100 (such as in step G).

FIGS. 2A-C depict example system components that can be used to performthe techniques described herein. Referring to FIG. 2A, as described inreference to FIG. 1 , the centralized hub 102, sensor devices 108A-N,and sensors 112A-N can communicate via the network(s) 114. Thecentralized hub 102, sensor devices 108A-N, and sensors 112A-N can alsocommunicate with a security system 200 and building information datastore 204 via the network(s) 114.

The sensor devices 108A-N can be configured to passively monitoranomalous signals and other conditions in a building, as describedthroughout this disclosure. To do so, the sensor devices 108A-N caninclude a suite of sensors and other components, including but notlimited to processor(s) 214, light sensor 216, sound sensor 218,temperature sensor 220, motion sensor 222, image sensor 224, outputdevice(s) 226, communication interface 228, and power source 229. Theprocessor(s) 214 can be configured to perform one or more techniques andoperations described herein. For example, sometimes, one or more of thesensor devices 108A-N can be configured to operate like the centralizedhub 102. In other words, the one or more sensor devices 108A-N canrequest and/or receive detected signals from other sensor devices 108A-Ncan determine whether any of the detected signals exceed expectedthreshold values.

The one or more sensor devices 108A-N can then identify a security eventin the building. The sensors 216, 218, 220, 222, and 224 of the sensordevices 108A-N are further described in reference to FIG. 8A. Asmentioned throughout, some sensor devices 108A-N can include some of thesensors 216, 218, 220, 222, and 224 but not all. Moreover, some sensordevices 108A-N can include some of the sensors 216, 218, 220, 222, and224 while other sensor devices 108A-N include different sensors 216,218, 220, 222, and 224. For example, one of the sensor devices 108A-Npositioned at a front entrance to a building can include the lightsensor 216, the sound sensor 218, the motion sensor 222, and the imagesensor 224. Another sensor device 108A-N positioned in a kitchen of thebuilding can include the light sensor 216, the temperature sensor 220,and the image sensor 224, while another sensor device 108A-N positionedin a bedroom of the building may include only the light sensor 216 andthe temperature sensor 220. A variety of other configurations of sensorsin each of the sensor devices 108A-N can be realized and installed in abuilding.

The output device(s) 226 of the sensor devices 108A-N can include audiooutput 230 and visual output 232. As described further below (e.g.,refer to FIG. 8A), the sensor devices 108A-N or the centralized hub 102can determine a preferred form of outputting notifications, guidance, orother information to building occupants. Audio output can be preferredin emergency situations where, for example, there is a fire andoccupants are unable to see visual guidance prompts through smoke andflames. Visual output can be preferred in some security event situationswhere, for example, there is a break-in and audio signals could attracta thief to a location of the building occupant(s), thereby putting thebuilding occupant(s) at more risk.

Sometimes, the audio output 230 can also act as an input device. Forexample, a building occupant can speak to the sensor devices 108A-N andask what is happening in the building (e.g., refer to FIGS. 5A-B). Theaudio output 230 can detect the occupant's voice and, based on therequest from the occupant, ping the other sensor devices 108A-N tocollect detected signals and determine whether any security event hasoccurred in the building. The audio output 230 can then outputinformation to the occupant about the current state of the building. Thevisual output 232 can also output information about the current state ofthe building. For example, the visual output 232 can project text orother visualizations depicting the current state of the building on oneor more surfaces proximate the building occupant. Images captured of aroom in the building can, for example, be projected on a wall near theoccupant. Video captured of a room in the building can also be projectedon the wall such that the occupant can watch, in real-time, what ishappening in the building. This feature therefore provides for ARinteraction between the building occupants, the sensor devices 108A-N,and the centralized hub 102 such that the building occupants can easilyand intuitively remain up to date about what is currently happening inthe building. As a result, building occupants can feel more comfortableand safe in the building since they can always talk to the sensordevices 108A-N and receive, as feedback, guidance about what theoccupants can do to feel more comfortable and/or safe in the building.

The communication interface 228 can be configured to providecommunication between the sensor devices 108A-N and the componentsdescribed throughout. The sensor devices 108A-N can also include thepower source 229, which can provide power to the sensor devices 108A-N.The power source 229 can be any type of power supply, including but notlimited to batteries, solar energy, and/or plug-in battery packs.Sometimes, the sensor devices 108A-N may not have the power source 229.Sometimes, the power source 229 can be rechargeable. Moreover, in theevent that an external power source powering the sensor devices 108A-Ngoes down, a local power source, such as the power source 229, can beautomatically activated to power the sensor devices 108A-N. As a result,the sensor devices 108A-N can continue to operate and function normallywith the local power source 229. In such scenarios, the power source 229can act as a backup power supply.

As mentioned herein, the centralized hub 102, the sensor devices 108A-N,and the sensors 112A-N can communicate with the security system 200. Thesecurity system 200 can be a separate system that is already installedin the building and/or installed at a same or later time as thecentralized hub 102. The security system 200 can provide for knownsafety and security monitoring. Integration of the security system 200with the disclosed techniques can provide for improved and moreeffective security monitoring in the building. The security system 200can include, for example, additional sensors, cameras, or other devicesthat continuously monitor different areas inside and/or surrounding thebuilding. Data collected by the security system 200 can be used by thecentralized hub 102 in order to more accurately detect security events,identify types of security events, and determine severity levels of thesecurity events. The security system 200 can also include informationabout known types of security events and signals or conditions in thebuilding that can be indicative of different types of security events.The centralized hub 102 can use this information to more accuratelydetermine what type of security event has been detected in the building.

The centralized hub 102, as described throughout this disclosure, candetect security events, notify relevant parties about the detectedsecurity events, determine floor maps and escape plans for buildingoccupants, and generate egress guidance or instructions for buildingoccupants when security events or emergencies are detected. Accordingly,the centralized hub 102 can include an emergency egress module 206, asecurity module 208, and a communication interface 234. Thecommunication interface 234 can provide communication between thecentralized hub 102 and one or more of the components described herein.

The emergency egress module 206 of the centralized hub 102 can beconfigured to determine floor maps of the building, escape plans, andguidance to assist building occupants in safely and calmly egressingduring an emergency or security event. In reference to FIG. 2B, theemergency egress module 206 can include a building layout determiner236, an emergency identification module 238, an emergency guidancedeterminer 240, and an output generator 242.

The emergency egress module 206 is described further in reference toFIG. 2C. Briefly, the building layout determiner 236 is configured togenerate floor maps of the building. These floor maps can be saved asbuilding layouts 210A-N in the building information data store 204(e.g., refer to FIG. 2A). The building layout determiner 236 can alsopredict different emergency scenarios using the generated floor maps anddetermine safe ways for occupants to escape the building in thepredicted emergency scenarios. Prediction of emergency scenarios can beperformed with artificial intelligence (AI) techniques, processes,and/or algorithms, predictive analytics, and/or one or more machinelearning models. Thus, the building layout determiner 236 can generateegress or escape plans for the building based on the building floor mapsand/or additional information, such as information or characteristicsabout building occupants (e.g., age, agility level, disabilities, etc.)that can be received from the security system 200 and/or the userdevices 252A-N. Generated egress plans can also be stored in thebuilding information data store 204.

During runtime, the emergency identification module 238 can beconfigured to identify when an emergency occurs in the building. Theemergency identification module 238 can, for example, communicate withthe sensor devices 108A-N and the sensors 112A-N to receive real-timereadings of conditions in the building. Based on these readings, theemergency identification module 238 can determine when and if anemergency, such as a fire, gas leak, etc., occurs. Sometimes, thesecurity module 208 can identify a variety of security events that caninclude emergencies. When the security module 208 detects an emergency,the security module 208 can transmit a notification indicating thedetected emergency to the emergency identification module 238.Sometimes, when the security module 208 detects a security event, suchas a break-in, the security module 208 can notify the emergency egressmodule 206 such that the emergency egress module 206 can provideappropriate guidance to help the building occupants escape the building.

The emergency guidance determiner 240 can generate escape plans beforean emergency occurs in the building. The emergency guidance determiner240 can also select predetermined escape plans during runtime, when anemergency is detected. The emergency guidance determiner 240 can selectescape plans based on current locations of building occupants relativeto a location of the emergency and information about the buildingoccupants, such as their age, agility level, disability, etc.

The output generator 242 can be configured to select an optimal form ofoutputting guidance to the building occupants during an emergency. Thisselection can be made based on current conditions in the building. Forexample, if there is a fire and a building occupant is detected to be ina room with a significant amount of smoke, the output generator 242 canselect audio guidance as the preferred form of output for thisparticular building occupant. As another example, if the buildingoccupant is deaf, then the output generator 242 can select visualguidance, such as strobe lights, flashing lights, or other signals thathelp direct the building occupant to safety. The output generator 242can also determine whether to provide the output at the sensor devices108A-N and/or the user devices 252A-N. Such a determination can be madebased on building occupant's preselected preferences and/or locations ofthe building occupants relative to the emergency and/or the sensordevices 108A-N.

Still referring to the emergency egress module 206, FIG. 2C is aconceptual diagram depicting the emergency egress module 206 inoperation, when it automatically determines a floor map of the buildingand uses the floor map for advising occupants how to exit the buildingduring a detected emergency. The emergency egress module 206 can monitorthe building (e.g., a home) and can communicate with various deviceswithin the building using one or more wired and/or wireless network(s)114. For example, the emergency egress module 206 can communicate with aset of user detection devices 202A-N that can detect a presence andmovement of occupants within the building, at particular locationswithin the building. The user detection devices 202A-N can provide userpresence information 264 to the emergency egress module 206. The userdetection devices 202A-N can be of various configurations, such asmotion sensors, cameras, door sensors, window sensors, door locks andwindow locks, other security devices, or sensors 112A-N, etc.

The building layout determiner 236 included in the emergency egressmodule 206 can use the user presence information 264 to determine andstore a floor map 262 of the building (e.g., the building layout 210A-Nin the building information data store 204 depicted in FIG. 2A). Thefloor map 262 indicates routes within the building, including exits outof the building.

One or more emergency detection devices 112A-N (e.g., the sensors112A-N) can also be located within the building to detect differenttypes of emergencies, such as fires and/or gas leaks. The emergencydetection devices 112A-N can be of various configurations, such as asmoke detector and a heat sensor (e.g., a temperature sensor, aninfrared sensor, etc.).

The emergency egress module 206 can receive emergency indicationinformation 266 from the emergency detection devices 112A-N that mayindicate one or more locations within the building that may have anemergency. The emergency identification determiner 238 in the emergencyegress module 206 can determine whether the emergency indicationinformation 266 indicates the presence of an emergency in the building.The emergency indication information 266 can be temperature readings forexample.

The building layout determiner 236 can determine, in response to theemergency identification determiner 238 determining presence of theemergency, one or more exit routes 260 that can be used by occupants toexit the building, based on the floor map 262 and the emergencyindication information 266. Sometimes, as described in reference to FIG.2B, the emergency guidance determiner 240 can determine the one or moreexit routes 260. The determined exit routes 260 can be selected so at toavoid the locations within the building that may have the emergency(e.g., the emergency indication information 266 can indicate location(s)of single or multiple emergencies within the building).

The output generator 242 can generate and send signaling instructions268 to the sensor devices 108A-N located in the building, for the sensordevices 108A-N to emit signal(s) to indicate to occupants the determinedroutes 260 out of the building. The emitted signals can be voicecommands, lighted signals, or other signals, as described in more detailbelow.

The emergency egress module 206 can further include a system improvementengine 256. The improvement engine 256 can employ machine learning toimprove various functions of the emergency egress module 206, such asfunctions performed by the building layout determiner 236, the emergencyidentification determiner 238, and the output generator 242. In someembodiments, the improvement engine 256 is configured to include one ormore engines that are separate from the other modules or engines (e.g.,the building layout determiner 236, the emergency identificationdeterminer 238, and the output generator 242) in the emergency egressmodule 206. Alternatively, the modules or engines of the emergencyegress module 206 can be configured to operate or otherwise performfunctions of the improvement engine 256.

In some examples, the improvement engine 256 is configured to process aninput, such as the user presence information 264, and generate a floormap 262 based on the input. In addition or alternatively, theimprovement engine 256 is configured to process an input, such as thefloor map 262 and the emergency indication information 266, andgenerate, based on the input, one or more exit routes 260 which are usedto evacuate occupants in the building.

In some examples, the improvement engine 256 can be trained to generateand update the floor map 262 adapted to changing aspects of occupancy byresidents as well as visitors of a building. Further, the improvementengine 256 can be trained to predict presence and whereabouts ofoccupants in a building in response to determination of presence of anemergency and generate at least one exit route 260.

The improvement engine 256 operates to bolster functioning andeffectiveness of the emergency egress module 206 by adjusting the module206 for changing circumstances in occupant status or occasions withguests. As such, an egress plan for a building can be modified rapidlywith changing occupant circumstances including guest visitation. Such aplan can be stored locally in the module 206 and in a cloud forredundancy. In certain examples, biometric sensing devices are alsoemployed in implementing the improvement engine 256 and/or otheroperations of the module 206.

In addition to real-time exit route guidance, a general building escapeplan can be automatically created, based on automatically-determinedfloor maps, and made available for occupants, to view and rehearse,before emergencies occur. Generated plans can be stored both locally inthe emergency egress module 206, as well as in the cloud for failsaferedundancy. Generated plans can be viewed and maintained by occupants.For example, occupants can update a plan, such as when changes inoccupancy occur over time, or when changes in family membership occur(e.g., newborns, grown children leaving the household, deaths, and soforth), and occasions with babysitters or house guests in the home.

Although the emergency egress module 206 is described in reference toemergencies such as fires and/or gas leaks, the emergency egress module206 can perform similar or same functions when the security module 208detects a security event, such as a break-in, theft, and/or burglary.Accordingly, the emergency egress module 206 can determine appropriateand safe escape plans for building occupants during the detectedsecurity event.

Referring back to FIG. 2B, the security module 208 can be configured todetect security events in the building during runtime and to generateoutput that can notify building occupants or other relevant stakeholdersof the detected security events. The security module 208 can include anormal conditions determiner 244, a voice learning module 246, asecurity event detector 248, and an output generator 250.

The normal conditions determiner 244 can be configured to learndifferent states or conditions of the building. The normal conditionsdeterminer 244 can use one or more machine learning trained models toidentify and determine normal values for signals that are detectedthroughout the building. The normal conditions determiner 244 canreceive detected signals from the sensor devices 108A-N over certainperiods of time. For example, the determiner 244 can collect signalsover several 24 hour windows of time in order to identify what sounds,lights, motion, and/or visuals are typical during a normal day in thebuilding. The determiner 244 can also collect signals during particulartime periods, such as from Sam to 10 am every morning for 5 consecutivedays.

The received signals can be fed into one or more machine learning modelsas input. Output from the models can be indications and/or ranges ofexpected values for different signals that may be detected in thebuilding. Sometimes, the output from the models can also indicate orotherwise classify the signals that may normally be detected in thebuilding. For example, if audio signals are barely detected between thehours of 11 pm and Sam on most days, the normal conditions determiner244 can determine that the building occupants likely start waking up andgetting ready for the day after 8 am. As another example, if visualsignals from a camera near a front door of the building shows childrenleaving the house at 8:30 am on most days and returning around 4:30 pm,then the normal conditions determiner 244 can determine that thebuilding occupants are likely gone from the building during that 8 hourwindow of time. Therefore, any anomalous signals detected during that 8hour window of time on most days may be indicative of a security event.As yet another example, if temperature signals positioned in a kitchenof the building indicate increased temperature readings around 7 pm atnight on most days, then the normal conditions determiner 244 candetermine that the building occupants likely cook their dinners duringthat time most evenings. Therefore, abnormal increases in temperatureduring other periods of time during the day can be indicative of somesecurity event and/or emergency.

The normal conditions determiner 244 can also use one or more machinelearning models to continuously learn and modify the normal conditionsof the building. The normal conditions of the building can be updated toreflect temporary changes and also long term changes. For example, therecan be days where guests are invited in the building and detectedsignals are abnormally high relative to a historic spread of detectedsignals during similar or same periods of time. The normal conditionsdeterminer 244 can receive detected signals from a variety of sensordevices 108A-N and correlate those signals in order to determine thatthe abnormally high signals are due to a dinner party rather than asecurity event. The machine learning models can be trained to classifyand/or correlate different types of signals with different categories ofactivities that constitute normal conditions for a building. Forexample, during a dinner party, audio signals can be much higher thantypical audio signals. The audio signals during the dinner party canalso continue over a long period of time, such as several hours. Asecurity event, such as a break-in, can have higher audio signals thanthe typical audio signals, however such audio signals would occurbriefly over a short period of time, such as several seconds and/orminutes. The machine learning models can be trained to identifymagnitude and duration of changes in detected signals, such as the audiosignals described above, and classify the detected signals based on theidentified magnitude and duration of change. In the example of thedinner party, since the detected audio signals are consistently high forseveral hours, the machine learning models can identify that thedetected audio signals are associated with activity that does notconstitute a security threat.

Sometimes, the normal conditions determiner 244 can also receive inputfrom building occupants about changes to the building. This input can befed into the machine learning models in order to improve or otherwiseupdate the normal conditions for the building. The input can indicatetemporary changes to a current state of the building. Examples includeovernight guests, the building occupants going on vacation for a certainamount of time, a babysitter spending a couple hours there, the buildingoccupants leaving for a day-long excursion/activity, a party, etc. Theinput can also indicate more long term changes to the current state ofthe building. Examples include a dog or other pet becoming part of thebuilding household, a newborn child, an occupant dying, a childbeginning to play and practice an instrument, a weekly meetup in thebuilding with guests from outside the building household, a particularwindow of time every week when all the building occupants are away fromthe building, etc.

The input can be received from mobile devices of the building occupants,such as the user devices 252A-N. For example, the building occupants canprovide updates to the normal conditions of the building via a userinterface in a mobile application. The mobile application cancommunicate with the centralized hub 102 and/or the sensor devices108A-N to provide the building occupants with information about currentconditions in the building. The input can also be provided by theoccupants directly to the centralized hub 102 and/or the sensor devices108A-N. For example, a building occupant can speak to the sensor device108A. The occupant can tell the sensor device 108A that changes havebeen made to the building. The sensor device 108A can relay (e.g.,transmit) the occupant's verbal message to the security module 208 ofthe centralized hub 102, where the verbal message can be dissected(e.g., by the voice learning module 246) and used to update normalconditions for the building (e.g., by the normal conditions determiner244). The centralized hub 102 and/or the sensor devices 108A-N can alsoinclude a user interface presented via input and output devices such asa touchscreen display where the occupants can not only view informationabout the current conditions in the building but also provideinformation to update the normal conditions in the building.

In determining the normal conditions of the building, the normalconditions determiner 244 can average the signals in order to determinea baseline or threshold value for the signals. The baseline or thresholdvalue for the signals can indicate normal conditions for the buildingduring the designated time period. The normal conditions determiner 244can also find a median and/or mean value for the signals from adesignated time period in order to identify the normal conditions forthe building. Therefore, the normal conditions determiner 244 can take ahistoric spread of detected signals in order to determine what is normalfor the building.

Learning the normal conditions for the building can be advantageous touse as a baseline in determining whether detected anomalous signals areout of the ordinary and indicative of a security event. For example, ifa loud crashing sound is detected at 8 am in the morning and thehistoric spread of sound signals indicates that minimal noise isdetected between 5 am and 9 am, then the security module 208 candetermine that the loud crashing sound is likely a security event.Correlating various signals that are detected at a same time as the loudcrashing sound can be performed by the security module 208 in order toconfirm that a security event in fact occurred and that the loudcrashing sound wasn't some random activity, such as a book falling off ashelf.

Still referring to the security module 208 in FIG. 2B, the voicelearning module 246 can be configured to learn voices of buildingoccupants and to respond to verbal requests made by the buildingoccupants. The voice learning module 246 can use one or more machinelearning models and/or artificial intelligence (AI) techniques,processes, and/or algorithms to identify and associate voices withdifferent occupants in the building. The voice learning module 246 canalso use the machine learning models to identify different tones orinflections in the voices that can indicate how the occupants arefeeling. For example, if the voice learning module 246 detects anoccupant speaking fast, in a high pitched tone, and/or quietly to one ormore of the sensor devices 108A-N, then the voice learning module 246can determine, by applying one or more machine learning models, that theparticular occupant is feeling uncomfortable, concerned, and/or unsafe.Determining how the occupant is feeling can be beneficial to providemore accurate guidance, information, or other types of prompts to theoccupant.

As an illustrative example, if the occupant asks one of the sensordevices 108A-N whether everything is okay in the building, the voicelearning module 246 determines that the occupant is speaking in apanicked, rushed tone, and the security module 208 determines that asecurity event has occurred at the building, then the security module208 can determine that the occupant should be provided morecomprehensive, step-by-step guidance to help ease the occupant's panicand discomfort. As another example, the occupant can speak to one of thesensor devices 108A-N by asking for help (e.g., asking for a safest wayto exit the building during an emergency and/or saying that they do notknow what to do during an emergency) and based on the detected toneand/or inflection in the occupant's voice, the security module 208 candetermine an appropriate amount of guidance or other information toprovide to the occupant.

The machine learning models can be trained to identify how the occupantis feeling based on speed at which the occupant speaks, magnitude ofspikes in pitch of the occupant's voice, loudness of the occupant'svoice, whether the occupant speaks in fluid sentences or whether theoccupant's words are choppy or less fluid, etc. Therefore, the machinelearning models may identify how the occupant is feeling based oncharacteristics of the way the occupant speaks rather than actualcontent or things that the occupant is saying. In other words, the voicelearning module 246 may not be actively monitoring what is said by theoccupant but rather can passively monitor changes in the way that theoccupant speaks in order to determine whether the occupant iscomfortable or concerned about current conditions in the building.

Moreover, the voice learning module 246 can use the machine learningmodels to learn normal tones or ways of speaking for the buildingoccupants. Therefore, when the occupant speaks to one of the sensordevices 108A-N and asks for updates on the conditions in the building orasks for guidance, the voice learning module 246 can determine whetherthe occupant's voice deviates from a known, normal way that the occupanttypically speaks. A greater deviation can indicate that the occupant isfeeling uncomfortable, insecure, panicked, etc. A greater deviation cantherefore indicate that the occupant should be provided with moreinformation or guidance to help calm the occupant.

Still referring to the security module 208 in FIG. 2B, the securityevent detector 248 can be configured to identify a security event fromanalyzing anomalous signals that are detected by the sensor devices108A-N and/or the sensors 112A-N. As described in reference to FIGS. 1,3, and 4A-B, the security event detector 248 can receive detectedsignals from the sensor devices 108A-N and/or the sensors 112A-N. Thesecurity event detector 248 can retrieve threshold conditions 212A-Nfrom the building information data store 204 (e.g., refer to FIG. 2A).The threshold conditions 212A-N can be determined by the normalconditions determiner 244 and can be based on room or location in thebuilding where the sensor devices 108A-N are located. The thresholdconditions 212A-N can also be based on general areas or portions of thebuilding (a first floor, a second floor, a kitchen and dining room, aliving room, etc.) and/or the building as a whole. The thresholdconditions 212A-N can include light, sound, temperature, motion, and/orvisual for each of the designated areas, rooms, locations, and/or sensordevices 108A-N.

As described above, the threshold conditions 212A-N can indicateexpected or otherwise normal signal levels for the building (e.g.,expected amounts of light in a particular room, expected temperaturevalue and/or temperature range in a kitchen, expected motion or movementnear one or more entrances to the building, expected amount of noise ina hallway or other room, etc.). The security event detector 248 candetermine whether any of the received signals deviate from the thresholdconditions 212A-N as well as a magnitude and/or duration of thedeviation. Based on this analysis, the security event detector 248 canidentify whether a security event has occurred. Sometimes, the securityevent detector 248 can also identify that an emergency, such as a fire,has occurred based on a comparison of the received signals to thethreshold conditions 212A-N.

The security event detector 248 can also correlate different anomaloussignals received at a same or similar time in order to positivelyidentify that the security event occurred. Therefore, the security eventdetector 248 can improve its accuracy in detection of security events,thereby becoming more reliable to building occupants. Using one or moremachine learning models, the security event detector 248 can classifythe detected security event. For example, the models can be trained toidentify that combinations of certain types of signals and/or deviationsin such signals are representative of different types of emergencies. Asan illustrative example, the models can be trained to identify abreak-in based on a combination of the following signals: a sudden,sharp increase in decibel signals, a sudden increase in movement/motionnear a window, door, or other entrance into the building, and video orimage feed data depicted shattered glass, debris, or a foreign objectentering the building. As another example, the models can be trained toidentify afire based on a combination of the following signals: a suddenand large increase in temperature, detection of smoke, and sporadicshifts in lighting.

The security event detector 248 can also determine a severity level ofthe detected security event. One or more machine learning models can beapplied to the received and correlated signals in order to assess howsevere of a threat the security event poses to building occupants. Thesecurity event detector 248 can also determine the severity level basedon presence of the occupants in the building. For example, the modelscan be trained to determine that the security event is of lower severitywhen none of the occupants are detected as being present in thebuilding. Occupant presence can be determined based on user presenceand/or motion sensors positioned throughout the building. Occupantpresence can also be determined based on one or more of the sensordevices 108A-N outputting audio in the building that asks or otherwiseprompts present occupants to say something. Occupant presence can alsobe determined from video and/or image feed data captured by the sensordevices 108A-N, the sensors 112A-N, and/or the security system 200.Moreover, occupant presence can be determined based on pinging, by thecentralized hub 102, the user devices 252A-N of the occupants todetermine whether the devices 252A-N are located within the buildingand/or connected to a local home network.

Once the severity level is determined, the security event detector 248and/or the output generator 250 can determine appropriate guidance orother information to provide to the building occupants. The higher theseverity level, the more thorough and/or step-by-step guidance can beprovided to the building occupants. The lower the severity level, theless information can be provided to the building occupants.

Accordingly, the output generator 250 can determine how much informationto provide to building occupants, what form of output should be used forthat information, and what information should be outputted. For example,if the voice learning module 246 detects that the occupant is panicking,the output generator 250 can determine that more thorough informationshould be provided to the occupant in order to calm them down. Asanother example, if the security event detector 248 determines that aburglary has occurred, the output generator 250 can determine that theoccupant should receive a notification that there is a burglary andinstructions to safely, calmly, and quietly exit the building (e.g.,based on and/or using one or more of the escape plans generated by theemergency egress module 206).

The output generator 250 can determine an appropriate form of outputbased on the detected security event. If the security event involves abreak-in, for example, the output generator 250 can determine that audioand/or visual output may draw attention to the occupant and thereforeput the occupant's safety at increased risk. Thus, the output generator250 can determine that text messages and/or push notificationstransmitted to the user device 252A-N of the occupant can be a preferredform of output. As another example, if the security event involves afire, the output generator 250 can determine that audio that isoutputted by the sensor devices 108A-N can be preferred, especiallysince visual signals may not be seen in smoke and/or through flames. Asyet another example, if the centralized hub 102 is unable to connect toany of the user devices 252A-N of the occupants, then the outputgenerator 250 can select a form of output that can be provided by thesensor devices 108A-N positioned throughout the building. Similarly, ifthe centralized hub 102 is unable to connect to one or more of thesensor devices 108A-N, then the output generator 250 can select a formof output that can be provided by the user devices 252A-N of theoccupants.

Sometimes, the output generator 250 can also select a preferred form ofoutput based on information about the occupants. For example, if anoccupant is deaf, then the output generator 250 can generate visualsignals as output to be displayed by the sensor devices 108A-N (e.g.,flashing lights, strobe lights, directional signs projected onto walls,floors, windows, and/or doors, etc.). The output generator 250 can alsogenerate output, such as text messages and/or push notifications, thatcan be received and viewed by the deaf occupant at their user device252A-N.

The output generator 250 can select a different form of output for eachoccupant in the building. Output for some occupants can be visual whileoutput for other occupants can be audio. Moreover, output for someoccupants can be displayed by the sensor devices 108A-N while output forother occupants can be displayed by the user devices 252A-N.

As mentioned above, the output generator 250 can determine whatinformation should be provided to each occupant. What information toprovide can be based on tone of the occupant's voice when they speak tothe sensor devices 108A-N, as described above. What information toprovide can also be based on the type of security event, the severitylevel of the security event, and other information about the occupant.Sometimes, the output generator 250 can generate output that merelymentions that a security event was detected. Sometimes, the outputgenerator 250 can generate output that asks the occupant what type ofinformation they want/need. Based on the occupant's response, the outputgenerator 250 can generate additional or less guidance for theparticular occupant. Moreover, sometimes the output generator 250 cangenerate output that includes guidance to safely and calmly escape thebuilding. How much guidance to provide can be based on a variety offactors, as mentioned above, such as user preference, how the occupantis feeling, what type of security event was detected, and the severitylevel of the detected security event. Any information generated by theoutput generator 250 can then be transmitted to the sensor devices108A-N and/or the user devices 252A-N to be outputted to theoccupant(s).

Referring back to FIG. 2A, the user devices 252A-N can include mobiledevices such as smartphones, cellphones, laptops, computers, tablets,and/or wearable devices. Building occupants can have user devices 252A-Nthat can be in communication with components described herein, such asthe sensor devices 108A-N, the centralized hub 102, and/or the securitysystem 200. The user devices 252A-N can receive notifications (e.g.,push notifications at a mobile application), text messages, audiomessages, and/or other types of indications about a current state of thebuilding. Sometimes, for example, when a security event is detected, thecentralized hub 102 can send a push notification to the user devices252A-N of one or more building occupants to make them aware of thesecurity event. As another example, the centralized hub 102 can sendegress instructions to the user devices 252A-N of one or more buildingoccupants to safely guide such occupants to safety. The buildingoccupants can also use the user devices 252A-N in order to monitor thecurrent state of the building, whether the building occupants arelocated at the building or remote, and to ask the centralized hub 102 tocheck on the current state of the building.

FIG. 3 is a flowchart of a process 300 for detecting a security event ina building. The process 300 can be performed by the centralized hub 102described herein. More specifically, the process 300 can be performed byone or more components of the security module 208 of the centralized hub102 (e.g., refer to FIGS. 2A-B). One or more blocks of the process 300can also be performed by any of the sensor devices 108A-N, for example,where one of the sensor devices 108A-N acts or otherwise operates as thecentralized hub 102, when the centralized hub 102 goes down, and/or whenone of the sensor devices 108A-N identifies a large deviation in ananomalous signal that it detected. The process 300 can also be performedby a variety of other computer systems, networks of computers, servers,and/or devices. For illustrative purposes, the process 300 is describedfrom a perspective of a computer system.

Referring to the process 300, the computer system can receive signalsfrom sensors in a building in 302. As described in reference to FIG. 1 ,the computer system can receive the signals at predetermined times, suchas every 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, etc. Thecomputer system can also automatically receive the signals whenever asensor detects a change in state (e.g., a deviation in passivelymonitored signals) in the building. Moreover, the computer system canreceive the signals upon transmitting requests for any sensed signalsfrom the sensors. Also as described throughout this disclosure, thesensors can include the sensor devices 108A-N and/or the sensors 112A-Npositioned throughout the building. The receive signals can include butare not limited to audio (e.g., decibels), visual (e.g., video feeddata, image data), light, motion, temperature, and/or smoke signals.

The computer system can also retrieve expected threshold conditions in304. The computer system can retrieve from a data store expectedthreshold conditions for each of the received signals. For example, if alight signal is received from a sensor positioned in a kitchen of thebuilding, then the computer system can retrieve the expected thresholdcondition for light signals in the kitchen of the building. Moreover,the computer system can retrieve the expected threshold conditions for asame or similar timeframe as when the received signals were captured. Inthe example above, if the light signal is received at 9 pm, then thecomputer system can retrieve the expected threshold conditions for lightin the kitchen at or around 9 pm. Sometimes, the computer system canretrieve overall expected threshold conditions for the building. Theoverall expected threshold conditions can indicate an average of aparticular type of signal or combination of signals that represents anormal state or conditions of the building.

As described in reference to FIGS. 2A-C, the computer system can learnnormal conditions for the building. These normal conditions canestablish expected threshold conditions or ranges for different types ofsignals that can be sensed in the building. The expected thresholdconditions can be learned in a variety of ways. For example, theexpected threshold conditions can be learned using statistical analysisover time. The computer system can analyze signal values detected duringone or more periods of time (e.g., 8 am to 9 am every morning for acertain number of consecutive days). The computer system can average,for example, decibel levels during the one or more periods of time,identify spikes or dips in the decibel levels, and categorize thosespikes or dips as normal conditions or abnormal conditions. The computersystem can also use standard deviations from the historic spread ofdecibel levels in order to determine expected threshold conditions. Inso doing, the computer system can identify typical deviations from theexpected threshold conditions that may occur. Any deviation that exceedsthe identified typical deviations can be indicative of a security event.

The expected threshold conditions can also be determined and identifiedas static values rather than averages, standard deviations, and/orranges of values. Therefore, if the received signals ever exceed areally high static value in a short amount of time, the received signalscan be indicative of a security event. Thus, the computer system cananalyze a rate of rise in the received signals to determine whetherthese signals exceed expected threshold conditions for the building.

The expected threshold conditions can also be determined and identifiedbased on relativity. In other words, every few minutes, for example, thecomputer system can receive decibel signals. The computer system candetermine an average in decibel level. Over time, the computer systemcan determine whether the decibel level is increasing and a rate of risein decibel level relative to the average decibel level. Therefore, asudden and sharp increase in decibel level relative to the averagedecibel level during a short timeframe can be indicative of a securityevent.

In 306, the computer system can determine whether any of the receivedsignals exceed the respective expected threshold conditions beyond athreshold level. Sometimes, the computer system can combine the receivedsignals into a collective of signals. The computer system can thendetermine whether the collective of signals exceeds expected thresholdconditions beyond the threshold level. The threshold level can bepredetermined by the computer system and based on the type of signal, alocation where the signal was detected, a time of day at which thesignal was detected, and one or more factors about the building and/orthe building occupants. The threshold level can indicate a range ofvalues that, although deviate from the expected threshold conditions, donot deviate so much as to amount to a security event. The thresholdlevel can be greater in locations in the building where typically, or onaverage, there is more commotion or activity by the building occupants.The threshold level can be lower in locations in the building wheretypically, or on average, there is less commotion or activity by thebuilding occupants.

As an illustrative example, temperature signals can have a greaterthreshold level (e.g., a greater range of expected temperature values)in a bathroom where the temperature can drastically increase when anoccupant runs hot water in comparison to a bedroom, where an occupantmay only blast A.C. during summer months but otherwise maintain thebedroom at a constant temperature. Therefore, the temperature would haveto increase higher and faster in the bathroom than the bedroom in orderto trigger identification of a security event.

As another example, audio signals in a nursery can have a lowerthreshold level (e.g., a smaller range of expected sound) wheretypically a young child sleeps in comparison to a family room, which canhave a greater threshold level (e.g., a larger range of expected sound)since the occupants typically spend time there, talk, watch TV, andotherwise make a significant amount of noise there. Thus, a lesserdeviation in audio signals detected in the nursery can triggeridentification of a security event in comparison to the same deviationin audio signals being detected in the family room.

As yet another example, the expected threshold conditions for anywherein the building can change depending on the time of day. Duringnighttime hours when the occupants are typically asleep, any type ofsignal can have a lower threshold level than during daytime hours whenthe occupants are going about their daily activities in the building. Inother words, during the nighttime, signals that deviate slightly fromthe expected threshold conditions can be indicative of a security eventwhereas the same slight deviation may not be indicative of a securityevent during daytime hours.

If none of the signals exceed the respective expected thresholdconditions beyond the threshold level, then the signals likely indicatenormal or otherwise expected conditions in the building. There likely isnot a security event. The computer system can return to block 302.

If any of the signals exceed the respective expected thresholdconditions beyond the threshold level, then the computer system canidentify any other signals that were captured at a similar or same timeas the signals that exceed the respective expected threshold conditionsin 308. The computer system can then link the identified signals into asecurity event in 310. Thus, in 308 and 310, the computer system canverify that a security event in fact occurred based on analysis of othersignals detected throughout the building.

For example, if an audio signal detected at a front of the buildingexceeds the respective expected threshold condition beyond the thresholdlevel, then the computer system can also identify an audio signaldetected at a back of the building at the same or similar time as theaudio signal detected at the front of the building. If the audio signaldetected at the back of the building represents a deviation from theexpected threshold conditions, albeit a lesser deviation than that ofthe audio signal detected at the front of the building (e.g., sincesound can be more muted farther away from a location of an incident),then the computer system can confirm that the audio signal detected atthe front of the building likely constitutes a security event.

The computer system can also identify different types of signals in 308that can be linked into a security event in 310. For example, audio,light, and visual signals can be linked together to paint a story of asecurity event. One or more other signals can also be correlated orotherwise linked together to verify that the security event occurred andto depict what happened in the security event. Sometimes, the moresignals that can be linked to create a robust story of the securityevent can improve confidence and/or ability of the computer system todetect future security events. For example, the computer system can useone or more machine learning models to learn or otherwise improve thecomputer system's ability to detect security events from anomaloussignals. The security events identified in 310 can be provided as inputto the machine learning models in order to improve the computer system.

Next, the computer system can classify the security event in 312. Asdescribed in reference to FIGS. 2A-C, the computer system can apply oneor more machine learning models to the linked signals in order toclassify the security event. Classifying the security event can includedetermining a type of security event (314). The computer system can betrained to identify a variety of security event types from differentcombinations of signals and deviations in signals. The computer systemcan also be trained to identify the type of security event based on avariety of factors, such as how much the signals deviate from theexpected threshold conditions, what type of signals have been detected,where the detected signals were located in the building, whetheroccupants were present or near the signals when detected, etc.

Classifying the security event can also include determining a severitylevel of the security event (316). The severity level can be determinedbased on a variety of factors, including but not limited to the type ofsecurity event and how much the signals deviate from the expectedthreshold conditions. For example, a temperature signal of suchmagnitude and rapid rise from the expected threshold temperaturecondition for the building can indicate that a serious issue, such as afire, has begun in the building. This security event can be assigned ahigh severity level value. On the other hand, a temperature signal thatincreases enough to exceed the expected threshold temperature conditionover a longer period of time can be identified, by the computer system,as having a lower severity level. The severity level can be a numericvalue on a scale, such as between 1 and 100, where 1 is a lowestseverity level and 100 is a highest severity level. One or more otherscales can be realized and used in the process 300. The severity levelcan also be a boolean value and/or a string value.

The severity level can indicate how much the identified security eventposes a threat to the building occupants and/or the building. Forexample, a higher severity level can be assigned to the identifiedsecurity event when the event threatens safety of the buildingoccupants, protection of the occupant's personal property, and/orstructure of the building. As another example, a lower severity levelcan be assigned to the event when the event threatens a structure of thebuilding but the occupants are not currently present in the building.Therefore, threats that the security event poses can be weighed againsteach other in determining the severity level of the security event. Asdescribed in reference to FIGS. 2A-C, the severity level can bedetermined by the computer system by using one or more machine learningmodels.

Moreover, classifying the security event can include determining alocation of the security event (318). As described above, the computersystem can use one or more machine learning models to determine thelocation of the security event. The location can be determined based onidentifying strength of the received signals and proximity of thereceived signals from each other. For example, an audio signal receivedfrom a front of the building can greatly exceed threshold conditions forthe building while an audio signal received from a back of the buildingcan exceed threshold conditions by a smaller magnitude. The computersystem can be trained to identify that the security event likelyoccurred closer to the front of the building rather than the back of thebuilding. The computer system can then compare audio signals and othersignals received from locations proximate to the audio signal receivedfrom the front of the building in order to narrow down and pinpoint alocation of the security event. Using one or more building layouts(e.g., floor plans) and location information for the sensor devicespositioned throughout the building that detected the audio signals, thecomputer system can identify a room or other particular location in thebuilding where the security event occurred. Classifying the securityevent based on type, severity, and location can be beneficial todetermine appropriate guidance, instructions, or other information toprovide to building occupants and relevant stakeholders, such asemergency response personnel.

Once the security event is classified, the computer system can generateappropriate output in 320. Generating the output can include selectingan optimal form of output and determining what information to provide tobuilding occupants or other relevant stakeholders, such as emergencyresponse personnel. Selecting the optimal form of output can be based onthe type of security event. For example, if the security event isidentified as a burglary, the computer system can determine that audiooutput, whether provided by the sensor devices or the occupants' mobiledevices, can expose the occupants to the burglar and increase anassociated security risk. Therefore, the computer system can selectforms of output that include visual displays, text messages, and/or pushnotifications. As another example, if the security event is identifiedas a fire, the computer system can determine that visual output, asprovided by the sensor devices or other sensors in the building, may notbe preferred since smoke and flames can make it challenging for thebuilding occupants to view lighted signals. The computer system canselect forms of output that may include audio instructions or guidance.

As described throughout this disclosure, the output can includeguidance, instructions, prompts, or information that can be presentedvia audio, text messages, push notifications, and/or visuals that aredisplayed in the building. Sometimes the output can alert the buildingoccupant of a security event, such as the type of security event andwhere it occurred in the building. The output can also include guidanceor instructions to assist the building occupant to safely, calmly, andquickly escape the building and avoid the security event. Sometimes,depending on the type and severity level of the security event, theoutput can include a notification that is automatically transmitted toemergency response personnel, such as police, firefighters, EMTs, etc.Therefore, the building occupants can focus on their own safety andescape from the security event rather than figuring out who to contact.

Sometimes, the output can include questions that prompt the buildingoccupants to provide some sort of response. Depending on the occupants'response, the computer system can determine what additional informationcan be provided to the occupants. The computer system can use AR inorder to have conversations with the building occupants and act uponthose conversations. For example, the computer system can generateoutput that, when presented by a sensor device, asks the occupant howthey are feeling. The occupant can reply that they are concerned,feeling safe, scared, or any other variety of emotions they may beexperiencing. The sensor device can transmit the occupant's response tothe computer system for further analysis. As described in reference toFIGS. 2A-C, the computer system can learn the occupant's voice anddetermine a tone and/or inflection of the occupant's voice that canindicate how the occupant is feeling. The computer system can also useone or more semantic analysis techniques in order to detect how theoccupant is feeling.

If the computer system determines that the occupant is in fact feelingconcerned or scared, the computer system can generate additional outputintended to calm or otherwise assist the occupant is finding safety. Forexample, the computer system can contact emergency response personnelupon determining that the occupant is scared for their life. Thecomputer system can also generate step-by-step instructions to helpguide the occupant safely out of the building while avoiding a locationof the security event.

If the computer system determines that the occupant is not as concernedor is otherwise feeling relatively safe, the computer system candetermine that additional guidance may not be needed for the occupant.

Regardless of whether a security event is detected, the occupants in thebuilding can converse with the sensor devices. As described throughout,the occupants can ask the sensor devices for updates on current states(e.g., conditions) in the building. The occupants can also tell thesensor devices about updates to the building and/or occupants in thebuilding (e.g., a change in disability or athleticism, moving in of anoccupant, moving out of an occupant, birth of a child, death of anoccupant, hosting of a dinner party or other social event, going away ona vacation, etc.). Information that the occupant shares with the sensordevices can be transmitted to the computer system, analyzed, and used toupdate, modify, or improve stored information, emergency escape plans,building layouts, detection of security events, and generation ofinstructions or guidance.

Finally, generating the output in 320 can also include transmitting theoutput to the appropriate devices. The computer system can connect to ahome network and poll devices to determine which devices are present inthe home network. The computer system can poll devices such as thesensor devices and/or mobile devices of the building occupants. If thedevices are determined to be present in the home network, then thecomputer system can transmit the output locally. If the devices are notpresent in the home network (e.g., the home network goes down, theoccupants and their respective mobile devices are not currently locatedat the building, a sensor device is offline, etc.), then the computersystem can transmit the output over a different network, such as acellular network or WIFI.

FIGS. 4A-B is a flowchart of an example process 400 for detecting asecurity event from irregular audio signals. Although the process 400 isdescribed in reference to analysis of audio signals, the process 400 canbe performed to analyze a variety of other types of signals describedthroughout this disclosure, including but not limited to light, visual,temperature, and/or motion signals.

The process 400 can be performed by the centralized hub 102 describedherein. More specifically, the process 400 can be performed by one ormore components of the security module 208 of the centralized hub 102(e.g., refer to FIGS. 2A-B). One or more blocks of the process 400 canalso be performed by any of the sensor devices 108A-N, for example,where one of the sensor devices 108A-N acts or otherwise operates as thecentralized hub 102, when the centralized hub 102 goes down, and/or whenone of the sensor devices 108A-N identifies a large deviation in ananomalous signal that it detected. The process 400 can also be performedby a variety of other computer systems, networks of computers, servers,and/or devices. For illustrative purposes, the process 300 is describedfrom a perspective of a computer system.

Referring to the process 400 in both FIGS. 4A-B, the computer system canreceive audio signals from a sensor device in a building in 402. Referto FIGS. 1 and 3 for further discussion on receiving signals. Thecomputer system can retrieve expected normal audio conditions for thebuilding in 404. Refer to block 304 in FIG. 3 for further discussion onretrieving the expected normal audio conditions for the building.

In 406, the computer system can determine whether the received audiosignal exceeds the expected normal audio conditions beyond a thresholdlevel. Refer to block 306 in FIG. 3 for further discussion. If the audiosignal does not deviate from the expected conditions beyond thethreshold level, then the computer system can return to block 402 andcontinue to receive audio signals from one or more sensors in thebuilding, such as the sensor devices. After all, the audio signal candeviate from the expected conditions within a threshold range and stillbe considered normal. If, on the other hand, the audio signal deviatesfrom the expected conditions beyond the threshold level, then thecomputer system can identify a potential security event in the building(408). Thus, the received audio signal is abnormal for the building.

The computer system can accordingly ping other sensor devices and/orsensors in the building for signals captured at a similar time as theaudio signal in 410. The computer system can transmit notifications withtimestamps to each of the sensor devices and/or sensors. Thenotifications can request signals that were captured at the same orsimilar timestamp as that of the audio signal received in 402. Byrequesting signals from other sensor devices and/or sensors, thecomputer system can correlate signals to determine whether a securityevent in fact occurred. By correlating the signals, the computer systemcan also more accurately classify the security event.

The computer system can receive the other signals from the other sensordevices and/or sensors in 412. The other signals can be other audiosignals like the one that was received in 402, except the other signalscan be detected in other locations in the building. For example, if theaudio signal received in block 402 was detected at a front of thebuilding, then the computer system can receive an audio signal from asensor device located at a back of the building. The other signals canalso be any one or more of light, visuals, temperature, smoke, motion,etc.

The computer system can then retrieve expected normal conditions foreach of the other signals (414). The expected normal conditions can beretrieved as described in reference to block 404. Sometimes, thecomputer system can retrieve an aggregate expected normal condition forthe building that represents a combination of the other signals that arereceived in block 412.

In 416, the computer system can determine whether any of the othersignals exceed the respective expected normal conditions beyond athreshold level. Refer to block 406 for further discussion.

If none of the other signals exceed the respective expected normalconditions beyond the threshold level, then the computer system canreturn to block 402 and repeat the process 400. In other words, thecomputer system can continue to passively monitor the building via thesensor devices and/or sensors. The computer system can continue toreceive anomalous signals from the sensor devices and/or sensors thatrepresent different detected conditions in the building.

If any of the other signals exceed the respective expected normalconditions beyond the threshold level, then the computer system can linkthose signals with the audio signal 418. In other words, the computersystem can confirm or otherwise verify that a security event wasdetected. By linking or correlating the signals that exceed expectednormal conditions beyond the threshold level during a same or similartimeframe, the computer system can positively identify the securityevent.

As described herein, the computer system can classify the linked signalsas a security event (420). Classifying the security event can includeidentifying a type of security event, severity level of the securityevent, and location of the security event. Refer to block 312 in FIG. 3for further discussion.

FIG. 5A is a conceptual diagram of an example scenario where an occupant502A verbally requests an update on conditions in building 500 fromsensor device 108C. FIG. 5B is a conceptual diagram of another examplescenario where the occupant 502A verbally requests an update onconditions in the building 500 from the sensor device 108C. Referring toboth FIGS. 5A-B, the building 500 can be a home, such as a singlefamily, single story home. The building 500 can also be any other typeof home, building, or establishment that can implement egress advisementand safety security systems therein. For example, the building 500 caninclude an apartment building, a high rise building, a commercialbuilding, and/or a multi-story home. One or more other buildingconfigurations are possible.

The building 500 depicted in both FIGS. 5A-B includes three rooms: rooms504A, 504B, and 504C. Each of the rooms 504A, 504B, and 504C have one ormore sensor devices 108A-N. For example, the room 504A includes sensordevices 108B and 108B. The room 504B includes sensor devices 108A and108D. The room 504C includes sensor device 108C. The room 504C alsoincludes a door 506 that allows for occupants and other users in thebuilding 500 to enter and exit the building 500. Although not depicted,the building 500 can include one or more windows, additional sensors,additional or fewer sensor devices 108A-N, additional doors, additionalrooms, and/or additional floors (e.g., levels).

In the example scenario in FIGS. 5A-B, the sensor device 108C can beoperating as a centralized hub (e.g., refer to the centralized hub 102described throughout this disclosure). Sometimes, any of the sensordevices 108A-N can operate as the centralized hub. Moreover, each of thesensor devices 108A-N can perform functions of the centralized hub at asame time or at different times without any one of the sensor devices108A-N being designated or otherwise appointed as the centralized hub.

Referring to FIG. 5A, at time=1, the occupant 502A is located in theroom 504C. The room 504C can be an entrance room to the building 500. Adog 508 can also be located in the room 504C. At time=1, the dog 508 canbe exiting the building 500 through the door 506. Occupant 502B islocated in the room 504A, which can be a living room. The occupant 502Bcan be watching TV at time=1. Occupant 502C is in the room 504B, whichcan be a kitchen. The occupant 502C can be cooking a meal in the kitchenat time=1.

At time=1, the occupant 502A can ask the sensor device 108C ifeverything is okay in the building 500. For example, the occupant 502Acan speak out loud, “Clarus, is everything ok?” (510). The sensor device108C can detect the occupant 502A's voice (step A). As describedthroughout this disclosure (e.g., refer to FIGS. 2A-C), the sensordevice 108C can detect the occupant 502A's voice using voice recognitiontechniques. Using AI and/or AR, the sensor device 108C can determinewhat the occupant 502A is asking the sensor device 108C. For example,the sensor device 108C can be trained to understand or detect one ormore predetermined commands, such as “Clarus, is everything ok?” Overtime, the sensor device 108C can also learn additional and potentiallymore nuanced, less direct commands that are said by the occupant 502A orother occupants in the building 500. In this example, the sensor device108C can detect that the occupant 502A has spoken a known command, whichis asking for a status update on conditions in the building 500.Accordingly, the sensor device 108C can ping the other sensor devices108A-N in the other rooms 504A-504B for any signal updates. The sensordevice 108C can transmit a notification to the other sensor devices108A-N asking the other sensor devices 108A-N if they detected anyanomalous signals over some predetermined period of time. Thenotification can also request the other sensor devices 108A-N to providethe sensor device 108C with any signals that were detected over somepredetermined period of time. Moreover, the notification can request theother sensor devices 108A-N to detect and provide current signals in therooms 504A-B at time=1. Therefore, the sensor device 108A can determineor otherwise identify real-time conditions in the building 500.

Each of the other sensor devices 108A-N can detect signals in theirrespective rooms 504A and 504B (step C). As described above, the othersensor devices 108A-N can detect real-time signals in the rooms 504A and504B. The other sensor devices 108A-N can also identify signals in therooms 504A and 504B that had been detected during one or morepredetermined time periods that are requested by the sensor device 108C.

The other sensor devices 108A-N can then transmit any of the detectedsignals to the sensor device 108C (step D). Sometimes all the sensordevices 108A-N can transmit detected signals to the sensor device 108C,even if the detected signals are constant, minimal, or otherwisenonexistent. For example, sensor device 108D may sense no movement oraudio in a location proximate to the sensor device 108D in the kitchen.Although no movement or audio is detected, the sensor device 108D canstill transmit notification to the sensor device 108C indicating absenceof movement or audio. Sometimes, if any of the sensor devices 108A-N donot detect any signals, such sensor devices 108A-N may not transmitsignals or any types of notifications to the sensor device 108C in stepD. Thus, the sensor device 108C may only receive signals that arepositively detected by the sensor devices 108A-N or signals that aredetected beyond some threshold level.

For example, detected motion near an open window that can be attributedto a curtain shifting in a breeze coming through the window may notexceed the threshold level to then be transmitted to the sensor device108C. After all, this detected motion may not be serious enough towarrant any type of inquiry into whether the motion is related to asecurity event or an emergency. On the other hand, swift detected motionthrough the door 506 in the room 504C can be transmitted to the sensordevice 108C (if detected by another sensor device or sensor other thanthe sensor device 108C) because this movement can exceed the thresholdlevel. This movement can be significant enough to warrant inquiry intowhether the movement is related to a security event or an emergency.

In the example of FIG. 5B, the sensor device 108C can detect movementsignals in the room 504C. The sensor device 108C can detect the dog 508exiting the building 500 through the door 506. The sensor device 108Ccan also detect audio (e.g., decibel) signals that represent the dog508's footsteps as it exits the building 500 through the door 504C. Ifthe sensor device 108C has an image sensor, then the sensor device 108Ccan also capture image data of the occupant 502A standing near thesensor device 108C and asking the sensor device 108C if everything isokay in the building 500. The sensor device 108C can also capture imagedata of the dog 508 as the dog exits the building 500 through the door506.

In the room 504A, the sensor device 108B can detect audio signals fromthe TV that is located near the sensor device 108B. If the sensor device108B also includes an image sensor, then the sensor device 108B can alsocapture image data of the occupant 502B sitting in front of the TV andwatching TV. The sensor device 108B can also detect light signals thatemanate from the TV or from the occupant 502B turning off the lights inthe room 504A in order to see the TV better. The sensor device 108B canalso detect no or minimal movement from the occupant 502B, especially ifthe occupant 502B is sitting on the couch watching TV rather than movingaround.

The sensor device 108N may detect audio signals that are less inmagnitude or volume than the audio signals detected by the sensor device108B in the room 504A. The sensor device 108N can detect more muted orlower volume decibels that represent the same audio from the TV that wasdetected by the sensor device 108B. Moreover, like the sensor device108B, if the sensor device 108N has an image sensor, the sensor device108N can also capture image data of the occupant 502B watching the TV.

In the room 504B, the sensor device 108A can detect audio signals of theoccupant 502C moving pots and pans around the kitchen, kitchen utensilsand equipment clanging or otherwise making sounds, and/or alarms,clocks, or presets activating on certain kitchen equipment. The sensordevice 108A can also capture image data of the occupant 502C cooking atthe stove. The sensor device 108A can detect movement of the occupant502C as they navigate the kitchen and cook. Moreover, the sensor device108A can detect room temperature values near the stove. The sensordevice 108A can also detect smoke near the stove. Sometimes, the sensordevice 108A can also detect changes in light, such as lights turning onfor certain kitchen equipment (e.g., a stove light turning on, amicrowave light turning off, etc.).

The sensor device 108D can detect similar signals as the sensor device108A in the room 504B. For example, the sensor device 108D can detectaudio signals that are of lesser volume and/or magnitude than thosesignals detected by the sensor device 108A. This is true if the audiosignals originate from a location closer to the sensor device 108A andfarther away from the sensor device 108D. The sensor device 108D canalso detect different levels and/or magnitudes of movement, temperature,light, and/or image data signals that are also detected by the sensordevice 108A.

Sometimes, each of the sensor devices 108A and 108D can have differentsets of signal sensors in order to passively monitor the room 504B. Asan illustrative example, the sensor device 108A can have temperature,motion, and audio sensors and the sensor device 108D can have light andimage sensors. In this configuration, the sensor device 108D can capturea full view of the kitchen to detect more accurate light and/or imagedata signals. Likewise, since the sensor device 108A is positionedcloser to kitchen equipment such as the stove, the sensor device 108Acan capture more accurate temperature, motion, and/or audio signals thanthe sensor device 108D. Any other configurations and/or combinations ofsignal sensors are possible for the sensor devices 108A and 108D in theroom 504B as well as the sensor devices 108B and 108N in the room 504A.

Still referring to FIG. 5A, the detected signals described above can betransmitted to the sensor device 108C in step D. The sensor device 108Ccan then determine whether any of the signals exceed expected thresholdconditions (step E). The sensor device 108C can determine whether any ofthe signals exceed the expected threshold conditions by one or morepredetermined amounts (e.g., levels, ranges, values).

As described in FIGS. 3-4 , the sensor device 108C can retrieve theexpected threshold conditions for signals that can be detected in eachof the rooms 504A-504C and/or the building 500 as a whole. The expectedthreshold conditions can be based on values that indicate a normal stateof each of the rooms 504A-504C and/or the building 500 as a whole at aparticular time that the occupant 502A asks the sensor device 108Cwhether everything is okay in the building 500.

As an example, the motion signal representing the dog 508 exitingthrough the door 506 of the building 500 may not be an expected amountof motion by the door 506 at the particular time that the occupant 502Awants to know the current state of the building 500. However, the motionsignal representing the dog 508's movement may not be significant enoughto exceed the expected threshold condition of movement near the door 506by a predetermined amount. In other words, if the dog 508 sprints outthe door 506, that detected motion can be significant enough to exceedthe expected threshold condition by the predetermined amount. However,if the dog 508 merely walks out the door 506, this movement may not beenough to exceed the expected threshold condition by the predeterminedamount, even if the dog 508 exiting through the door 506 is not a normalcondition at time=1.

The sensor device 108C can determine a status of the building 500 basedon determining whether any of the signals exceed the expected thresholdconditions (step F). As described in reference to FIGS. 1-4 , the sensordevice 108C can identify a security event when one or more of thesignals exceed the expected threshold conditions beyond thepredetermined threshold level(s). In the example of FIG. 5A, the sensordevice 108C can determine that everything is normal in the building 500.The sensor device 108C may not detect any security events oremergencies.

For example, in determining the status of the building 500, the sensordevice 108C can determine identify what is happening in each of therooms 504A-504C. Based on the combination of signals detected in theroom 504A, the sensor device 108C can determine that the occupant 502Bis merely watching TV and that no irregular activity has been identifiedin the room 504A. Similarly, the sensor device 108C can determine thatthe occupant 502C is merely cooking a meal at time=1 in the room 504B.Although a detected temperature value for the room 504B may be higherthan an average expected temperature for the room 504B at time=1, thedetected temperature value may not exceed the average expectedtemperature by the threshold amount. If the detected temperature valuedid exceed the average expected temperature by the threshold amount,then the sensor device 108C can determine that there may be an emergencyin the room 504B, which can be attributed to a cooking-related fire.Finally, the sensor device 108C can determine that the dog 508 hasexited the room 504C and gone outside, which, although may not be anexpected movement at time=1, may not be enough movement to exceed theexpected movement by the threshold amount.

The sensor device 108C can output the status of the building (step G).For example, the sensor device 108C can output an audio message to theoccupant 502. An example audio message can be “Nothing is out of theordinary” (512). Using AI and/or AR, the sensor device 108C can detectconcern in the occupant 502A's voice when the occupant 502A initiallyrequests knowing the status of the building 500. If the sensor device108C detects concern, then the sensor device 108C can provide additionaloutput to the occupant 502A in step G in order to ease the occupant502A. For example, the sensor device 108C can output information aboutwhat is happening in each of the rooms 504A-504C. The sensor device 108Ccan tell the occupant 502A what each of the other occupants 502B and502C are doing at time=1.

Sometimes, when the sensor device 108C outputs the status of thebuilding in step G, the occupant 502A can ask follow-up questions. Thesensor device 108C can detect the occupant 502A's voice and, using AI,AR, and/or voice recognition techniques, can have a conversation withthe occupant 502A. Having a conversation with the occupant 502A caninclude completing operations or tasks that the occupant 502A asks ofthe sensor device 108C. Having a conversation with the occupant 502A caninclude providing the occupant 502A with audio output that responses tothe requests made by the occupant 502A.

As an illustrative example, after the sensor device 108C outputs,“Nothing is out of the ordinary” (512), the occupant 502A can ask, “Canyou tell me what is happening in the kitchen?” The sensor device 108Ccan detect this question and perform actions to generate a response tothe question. The sensor device 108C can access, from a data store, afloor map or other layout information for the building 500 (e.g., referto FIGS. 2A-C). The sensor device 108C can use the floor map todetermine which of the rooms 504A-504C is the kitchen and which of thesensor devices 108A-N are located within the kitchen. Thus, the sensordevice 108C can determine that the room 504B is the kitchen and thesensor devices 108A and 108D are located therein. The sensor device 108Ccan then transmit a notification to the sensor devices 108A and 108Dwith a request for any signals detected at a time (e.g., timestamp) thatthe sensor device 108C received the question from the occupant 502A. Thesenor device 108C can also identify the signals already received fromthe sensor devices 108A and 108D to determine what is happening in thekitchen, instead of requesting new signals from the sensor devices 108Aand 108D.

The sensor device 108C can receive the signals and accordingly determinewhat is happening in the kitchen. The sensor device 108C can use AIand/or one or more machine learning models to convert the signals intodescriptive events to then be outputted to the occupant 502A. If, forexample, the received signals are image data, then the sensor device108C can apply one or more machine learning models to the image data toidentify what type of activity is depicted in the image data. The imagedata can depict the occupant 502C cooking at the stove, which can beidentified by the sensor device 108C based on application of the one ormore machine learning models. Thus, the sensor device 108C can respondto the occupant 502A by outputting audio stating, “Someone is cooking inthe kitchen.” One or more other responses and/or types of responses canbe generated and outputted to the occupant 502A. For example, theoutputted response can be in the form of image data or another visual.The image data can be presented at the sensor device 108C. The imagedata can also be presented at a mobile device of the occupant 502A. Oneor more other variations of responses and output are possible.

FIG. 5B depicts the building at time=2. Time=2 can be later than time=1in FIG. 1 . Time=2 can also be unrelated to time=1 in FIG. 1 .

At time=2, the dog 508 is outside of the building 500, the occupant 502Ais still in the room 504C, the occupant 502B is still watching TV in theroom 504A, and the occupant 502C is still cooking in the room 504B.However, a fire 518 has started in the room 504B at the stove where theoccupant 502C is cooking. As described throughout this disclosure, oneor more of the sensor devices 108A-N, such as the senor device 108A inthe room 504B, can automatically detect an emergency or security event,such as the fire 518, then determine egress routes for each of theoccupants 502A, 502B, and 502C that avoid the detected emergency.However, in the illustrative example of FIG. 5B, the occupant 502 asksthe sensor device 108C in the room 504C, “Clarus, is everything ok?”(514).

As described in reference to FIG. 5A, the sensor device 108C can detectthe occupant 502A's voice (step A). Based on detecting the occupant502A's voice, the sensor device 108C can determine that the occupant502A wants an update on what is happening in each of the rooms 504A-504Cin the building 500. Accordingly, the sensor device 108C can ping thesensor devices 108A-N in each of the rooms 504A and 504B (step B). Eachof the other sensor devices 108A-N can detect signals in theirrespective rooms 504A and 504B (step C) and transmit such signals to thesensor device 108C (step D).

Once the sensor device 108C receives the signals from the other sensordevices 108A-N, the sensor device 108C can determine if any of thesignals exceed the expected threshold conditions (step E). If any of thesignals do exceed the expected threshold conditions by a thresholdlevel, then the sensor device 108C can identify an emergency (or othertype of security event) in the building (step F). Identifying theemergency can include classifying the emergency, identifying a locationof the emergency, and determining a severity level of the emergency(e.g., refer to FIGS. 1-4 ).

In the illustrative example of FIG. 5B, the sensor device 108C canreceive a temperature signal from the sensor device 108A in the room504B that is great in magnitude, value, and duration. In other words,the received temperature value can be significantly higher than anaverage expected temperature value for the room 504B and/or a previouslydetected temperature value for the room 504B (e.g., the temperaturevalue at time=1). The received temperature value can therefore exceedthe expected threshold condition beyond a predetermined threshold level.The received temperature value can also spike drastically from theprevious temperature value for the room 504B in a short period of time(e.g., in a timeframe that is less than some predetermined thresholdperiod of time). Moreover, the received temperature value can beconsistent for more than some predetermined period of time. Because thereceived temperature value can exceed the expected threshold conditionsbeyond the threshold amount, the sensor device 108C can determine thatan emergency, such as the fire 518, has been detected in the room 504B.

The sensor device 108C can receive one or more additional signals thatcan be used to determine that an emergency exists in the room 504B. Forexample, received audio signals can have spikes in decibels thatresemble the occupant 502C screaming, yelling, or otherwise panicking.The sensor device 108C can be trained using one or more machine learningmodels and/or AI to detect the concern in the occupant 502C's voice anddetect an emergency based on such audio signals. Other signals that canbe used to detect the fire 518 include smoke signals, motion signals,light signals, and/or image data.

Once the sensor device 108C identifies the emergency (step F), thesensor device 108C can generate emergency guidance for the occupants502A, 502B, and 502C that are currently located in the building 500(step G). As described throughout this disclosure (e.g., refer to FIGS.2A-C), the sensor device 108C can retrieve a floor map of the building500 and predetermined egress escape plans for each of the occupants502A, 502B, and 502C. Using one or more of AI, predictive analytics,and/or machine learning models, the sensor device 108C can determinewhich egress escape plans should be selected for each of the occupants502A, 502B, and 502C. Selection of the egress escape plans can be basedon information about each of the occupants 502A, 502B, and 502C (e.g.,age, agility, disability, etc.), locations of the occupants relative tothe identified emergency (e.g., egress escape plans can be selected thatdo not involve passing or going through a location of the emergency),locations of the occupants relative to each other (e.g., the closertogether the occupants are, the more likely an egress escape plan can beselected that lets the occupants escape the building together), apredicted or projected spread of the emergency (e.g., egress escapeplans that can become obstructed at some point by the spread of theemergency may not be selected), an amount of time remaining beforecertain egress escape plans can become obstructed by the emergency(e.g., egress escape plans that will become obstructed by the spread ofthe emergency while the occupant is passing a point along the egressescape plan may not be selected), etc.

Generating emergency guidance for the occupants 502A, 502B, and 502C canalso include selecting an optimal form of output for such guidance.Based on the detected signals from the sensor devices 108A-N (step C),the sensor device 108C can determine whether audio and/or visual signalscan be outputted to each of the occupants 502A, 502B, and 502C. Forexample, signals received from the sensor devices 108A and 108D canindicate that there is smoke in the room 504B, so visual signals andguidance may not be seen well by the occupant 502C. The sensor device108C can determine that audio guidance should be outputted by either ofthe sensor devices 108A and 108D and/or text messages, audio, or pushnotifications should be transmitted to and outputted at a mobile device524 of the occupant 502C. Signals received from the sensor devices 108Band 108N can indicate that the room 504A does not have any smoke orbright lights that would make visual signals challenging to see.Therefore, the sensor device 108C can determine that visual guidance canbe outputted by either of the sensor devices 108B and 108N in the room504A. The sensor device 108C can also determine that audio guidance canalternatively or additionally be outputted by either of the sensordevices 108B and 108N.

Once the sensor device 108C determines and generates appropriateemergency guidance, the sensor device 108C can transmit the guidance toone or more devices in the rooms 504A, 504B, and 504C where theoccupants 502B, 502A, and 502C are located (step H). Therefore, if aroom is empty (e.g., no occupants are detected by user detection ormotion sensors and/or the sensor devices 108A-N), then the guidance maynot be outputted at device(s) in that room.

As shown in the illustrative example of FIG. 5B, the sensor device 108Ccan respond to the occupant 502A by outputting a message that states, “Afire started in the kitchen. Evacuate through the door!” (516). Thesensor device 108B in the room 504A can output audio guidance to theoccupant 502B that states, “A fire started in the kitchen. Leave theroom immediately and avoid the kitchen.” Although not depicted, thesensor device 108B and/or 108N can also output visual signals such asflashing and/or colored lights that are projected on walls and/or floorsin a direction that the occupant 502B can follow to exit the building500. The sensor device 108A in the room 504B can output audio guidanceto the occupant 502C that states, “Leave the kitchen immediately andexit through the door” (522). Although not depicted, the sensor device108A and/or 108D can also output visual signals such as flashing and/orcolored lights that are projected on walls and/or floors in a directionthat the occupant 502C can follow to exit the building 500. Moreover, asshown in FIG. 5B, a notification can be received at the occupant 502C'smobile device 524. The notification can be a text message, audioguidance, and/or a push notification that prompts and guides theoccupant 502C to safely, quickly, and calmly leave the room 504B.

Sometimes, as the occupants 502A, 502B, and 502C egress from thebuilding 500, the sensor devices 108A-N can passively monitor thesituation to see if the fire 518 gets worse and/or whether the occupants502A, 502B, and 502C are panicking or otherwise struggling to egress.For example, the sensor devices 108A-N can use AI techniques to detectpanic (e.g., screams, whimpers, crying, etc.) amongst any of theoccupants 502A, 502B, and 502C from audio signals. If the sensor devices108A-N detect panic, then the sensor devices 108A-N can generate andoutput additional guidance intended to ease the panic and fear from anyof the occupants 502A, 502B, and 502C.

If the sensor devices 108A-N detect any changes to the fire 518 that mayaffect safe egress of the occupants 502A, 502B and 502C, the sensordevices 108A-N can generate and output information to that effect tonotify the occupants 502A, 502B, and 502C. Any of the sensor devices108A-N can also perform a single course correction by selecting orotherwise updating the egress escape plans that were selected for any ofthe occupants 502A, 502B, and 502C.

Moreover, based on passive monitoring of the fire 518, any of the sensordevices 108A-N can also determine if and when to send notification ofthe fire 518 to emergency response personnel, such as firefighters,police, and/or EMT. Sometimes, the sensor device 108C can transmit anotification of the fire 518 to the emergency response personnel uponidentifying the emergency in the building (step F). Sometimes, thesensor device 108C may only transmit the notification to the emergencyresponse personnel based on determining that a detected severity levelof the fire 518 exceeds some threshold reporting out level. Furthermore,the sensor device 108C can transmit the notification to the emergencyresponse personnel after guidance has been outputted to the occupants502A, 502B, and 502C and based on passive monitoring of the fire 518 asthe occupants 502A, 502B, and 502C are egressing from the building 500.

Sometimes, the sensor device 108C as shown in both FIGS. 5A-B may not beoperating as the centralized hub. Therefore, when the sensor device 108Cdetects the occupant 502A's voice, the sensor device 108C can transmitthe voice signal to a remote computing system, such as the centralizedhub 102 described throughout this disclosure. The remote computingsystem can then perform steps B and E-G instead of the sensor device108C performing such steps in FIG. 5A. Similarly, the remote computingsystem can perform any of steps B, and E-H in FIG. 5B. Sometimes,another sensor device in the building 500 can operate as the centralizedhub. Accordingly, the sensor device 108C can send the voice signal tothe sensor device that operates as the centralized hub. That sensordevice can then perform steps B and E-G in FIG. 5A and/or steps B andE-H in FIG. 5B.

FIG. 6 depicts example features in a building that include sensordevices 108A-N integrated therein. As described throughout thisdisclosure, the sensor devices 108A-N can include a suite of one or moresensors that can be used to detect conditions in the building. Morespecifically, the suite of sensors can detect anomalous signals throughpassive monitoring in locations where the sensor devices 108A-N arepositioned throughout the building.

The sensor devices 108A-N can be integrated into one or more existingfeatures in the building. As a result, the sensor devices 108A-N can benonobtrusive. The sensor devices 108A-N also may not detract fromaesthetic appearances in the building. Sometimes, the sensor devices108A-N can also be standalone devices that can be configured in thebuilding, such as attached to walls, ceilings, placed in corners, and/orplugged into outlets. The sensor devices 108A-N that are standalonedevices can also be simple in design and size such that the sensordevices 108A-N do not appear obtrusive or diminish aesthetic appearancesin the building.

As shown in FIG. 6 , the sensor devices 108A-N can be integrated into avariety of existing features in the building. For example, sensor device108A can be integrated into a light switch 600. Integration of thesensor device 108A herein may not obstruct an ability to flip the lightswitch 600 on or to operate the light switch 600 as it is intended to beoperated. The sensor device 108A can then detect one or more types ofsignals in an area surrounding or otherwise proximate to the lightswitch 600.

Sensor device 108B can be integrated into a wall outlet 602. Integrationof the sensor device 108B herein may not obstruct an ability to plugpower cords into the outlets of the wall outlet 602. The sensor device108B can then detect one or more types of signals in an area surroundingor otherwise proximae to the wall outlet 602.

Sensor device 108C can be integrated into a thermostat 604 in thebuilding. Integration of the sensor device 108C herein may not obstructan ability to use any of the control options or display(s) of thethermostat 604. The sensor device 108C can then detect one or more typesof signals in an area surrounding or otherwise proximate to thethermostat 604.

Sensor device 108D can be integrated into an alert system 606 or otherexisting security system that is installed in the building. Integrationof the sensor device 108D herein may not obstruct an ability to use anyof the control options or display(s) of the alert system 606. The sensordevice 108D can then detect one or more types of signals in an areasurrounding or otherwise proximate to the alert system 606.

Sensor device 108N can be integrated into a lightbulb 608. Integrationof the sensor device 108N herein may not obstruct an ability to attachthe lightbulb 608 to a fixture or otherwise to use the lightbulb 608 forits intended purposes. The sensor device 108N can then detect one ormore types of signals in an area surrounding or otherwise proximate tothe lightbulb 608.

One or more other integrations of the sensor devices 108A-N withexisting building features are possible.

FIG. 7 depicts example forms of output that can be generated based on adetected security event. As described throughout this disclosure, thecentralized hub 102 and/or any of the sensor devices 108A-N can selectan optimal form of output to provide notifications to occupants in thebuilding. The centralized hub and/or the sensor devices 108A-N canselect the optimal form of output based on a variety of factors.

For example, output form can be selected based on occupant preferences.Before runtime, the occupant can provide input to the centralized hub102 indicating that the occupant is deaf and only wants to/can receivetext or push notifications to their mobile device and/or visual signalsoutputted by the sensor devices 108A-N in the building.

As another example, output form can be selected based on detection of anoccupant's mobile device in the building. If the mobile device isdetected in the building, guidance can be outputted there instead of bythe sensor devices 108A-N. This can be beneficial to protect theoccupant in situations where a thief or other assailant is in thebuilding since output by the sensor devices 108A-N can bring the thief'sattention to a location where output is proved by the sensor devices108A-N, thereby risking the safety of the occupant.

Output form can also be selected based on location of the occupantrelative to the emergency or security event. If the occupant is next toa fire, the occupant may respond quicker and more calmly to loud audiosignals rather than flashing lights.

Output form can also be selected based on severity level of theemergency or security event. If the emergency or security event is verysevere, such as a rapidly spreading fire, then output can be generatedand presented as audio and visual signals at every mobile device andsensor device 108A-N in the building. This can ensure that occupants seeand follow the guidance wherever they are in the building and that nooccupant is left behind. This type of output can also help the occupantsto understand the gravity of the situation, which can propel them toegress from the building quicker.

As another example, output form can be selected based on type of theemergency or security event. If the security event is a burglary,guidance can be outputted as text messages or push notifications to theoccupants' mobile devices in order to protect the occupants from beingfound or tracked down by the burglar. After all, output presented by thesensor devices 108A-N can attract the burglar's attention to locationsof the sensor devices 108A-N. If the occupants are located where theoutput is being presented, then the burglar can cross paths with theoccupants, thereby putting the occupant's safety at risk.

As yet another example, output form can be selected based on location ofthe occupant relative to sensor devices in the building. If the occupantis not located near any sensor device, then the guidance can betransmitted to the occupant's mobile device rather than the sensordevices 108A-N. On the other hand, if the occupant is located near anyof the sensor devices 108A-N and/or the occupant does not have theirmobile device on them, then the guidance can be outputted at the sensordevices 108A-N closest to the occupant.

Output form can further be selected based on ambient conditions wherethe occupant is currently located. If smoke is detected where theoccupant is currently located, guidance can be outputted as audiosignals instead of visual signals. After all, the smoke can prevent theoccupant from seeing the visual signals clear enough.

FIG. 7 depicts different forms of output that can be selected based onany of the factors listed above. For example, emergency guidance 702 canbe presented at a mobile device 700. The mobile device 700 can belocated in the building where the emergency or security event isdetected. The centralized hub 102 can, for example, detect that themobile device 700 is connected to a local home network. The emergencyguidance 702 can be presented at the mobile device 700 as a textmessage, audio message, and/or push notifications for a mobileapplication. Here, the example emergency guidance 702 states, “Break indetected! Exit your bedroom immediately through the window and do notmake much noise . . . the police have been notified and they shouldarrive soon.” The emergency guidance 702 can include additional or lessinformation, based on factors previously described throughout thisdisclosure (e.g., detected panic of the occupant, type of securityevent, severity level of the security event, etc.).

As another example, audio guidance 706 can be outputted by a signalingdevice 108A in building 704. The audio guidance 706 can state, “Firedetected in the kitchen! Exit through the door.” The audio guidance 706can be referring to door 712 in the building 704. As shown, a sensorstrip 710 can also be positioned above the door 712. The sensor strip710 can include a plurality of sensors, such as lights, that can beactivated by the centralized hub 102 to provide guidance to theoccupants when escaping the building 704. The sensor strip 710 can beany other type of sensor and/or signaling device(s), as describedfurther below in reference to FIGS. 8A-B.

As yet another example, instead of or in addition to the emergencyguidance 702 presented at the mobile device 700 and the audio guidance706 presented by the sensor device 108A, the sensor strip 710 can alsoilluminate to provide visual signals that guide the occupants to exitthe building 704. As described further below, the visual signalspresented by the sensor strip 710 can include flashing lights, onesteady light, arrows or other indications depicting a direction that theoccupants should be moving in, and/or X's or other indications depictinga direction that the occupants should be avoiding. The visual signalscan also include colors, such as green signals to demonstrate where itis okay for the occupants to go and red signals to demonstrate where theoccupants should not be going.

FIGS. 8A-B depict exemplary systems for providing emergency or securityevent detection, guidance and advisement. In FIG. 8A, sensor device 108Aof FIGS. 1-2 is used as an example. The sensor device 108A can be asingular device, as depicted in FIG. 8A, or it can optionally be spreadout physically with separate components that can be in wired or wirelesscommunication with each other (e.g., refer to FIG. 8B). In the exampleof FIG. 8A, the sensor device 108A includes a light signaling component830, an audio signaling component 840, and a sensor controller 852. Thesensor controller 852 can have a one-to-one ratio of communication.Alternatively, the sensor controller 852 can have a one-to-multipleratio of communication. The audio signaling component 840 and/or thelight signaling component 830 can optionally be integrated into/part ofa same housing unit and/or circuit board as each other, the sensorcontroller 852, and/or the sensor device 108A as a whole. Alternatively,each of the components in FIG. 8A, such as 830, 840, and 852, can behoused separately (e.g., separate devices, such as in reference to FIG.8B). The sensor controller 852 can sometimes be in the same housing withthe light signaling component 830 while the audio signaling component840 can be housed separately. Sometimes, the sensor controller 852 andthe audio signaling component 840 can share the same housingunit/circuit board while the light signaling component 830 is arrangedseparately. Moreover, sometimes the components 830 and 840 can be housedin the same unit and the sensor controller 852 can be housed separately.

In the example of FIG. 8A, the components 830 and 840 are housed in thesame unit as the signaling controller 852, in the sensor device 108A.Optionally, the sensor device 108A can have an external power supply 870(e.g., lithium battery). The sensor device 108A can also receiveemergency and/or security event signals from the centralized hub 102 orany of the other sensor devices 108A-N in a building, as describedthroughout this disclosure. The sensor controller 852 can communicatedirectly with the light signaling component 830 as well as the audiosignaling component 840 in order to provide guidance, instructions, orother information to building occupants about a detected emergency orsecurity event.

The sensor controller 852 can include a predetermined signaling logic854, a predetermined output logic 856, a temperature sensor 858, a userpresence sensor 860, a light sensor 866, a sound sensor 868, a motionsensor 872, an image sensor 874, and a communication interface 862. Thesensor controller 852 can optionally include a power source 864 (e.g.,battery) in order to power the sensor controller 852 and/or the sensordevice 108A. Sometimes, the sensor controller 852 may not have one ormore of the sensors 858, 860, 866, 868, 872, and 874, and instead cancollect sensor information from sensors or other sensor devices 108A-Npositioned throughout the building, as described throughout thisdisclosure.

The predetermined signaling logic 854 can select an optimal egressescape route from a list of predetermined egress plans during areal-time emergency or security event (e.g., refer to egress escape plangeneration by the emergency egress module 206 in FIGS. 2A-C). Thisselection can be based on information sensed in real-time by one or moreof the sensors 858, 860, 866, 868, 872, and 874, and/or any othersensors or signaling devices 108A-N positioned throughout the building.

Once an egress escape route is selected, the predetermined output logic856 can determine which form of output should be used to output theegress guidance, as described throughout this disclosure (e.g., refer tothe emergency egress module 206 and the security module 208 in FIGS.2A-C). Based on the determination of output form, the guidance can beoutputted using the light signaling component 830 and/or the audiosignaling component 840. Moreover, if the predetermined output logic 856determines that guidance should be outputted by an occupant's mobiledevice, then the communication interface 862 can transmit guidanceinstructions to the occupant's mobile device.

The sensors 858, 860, 866, 868, 872, and 874 are described throughoutthis disclosure (e.g., refer to FIGS. 1-3 and 5A-B). Briefly, thetemperature sensor 858 can be configured to detect changes intemperature in an area proximate to a location of the sensor device108A.

The user presence sensor 860 can determine whether an occupant islocated within a room. For example, the user presence sensor 860 candetect human movement (e.g., movement of a limb, walking), human bodytemperature, and/or human sounds (e.g., breathing, coughing). The userpresence sensor 860 can also be configured to detect whether anoccupant's mobile device (e.g., smartphone, cellphone, BLUETOOTHheadphones, wearable device, etc.) is connected to a local home network.Moreover, the user presence sensor 860 can detect location signals ofthe occupant's mobile device to determine whether the occupant islocated in the room and/or proximate to the location of the sensordevice 108A.

The light sensor 866 can detect changes in light in an area proximate tothe location of the sensor device 108A. This can include changes inlight from lights being turned on and off, lightbulbs suddenly goingout, doors being opened or closed, windows being covered, uncovered,and/or broken, movement of furniture or objects that can obstruct lightin areas of the room, burning flames from a fire, etc. The light sensor866 can detect different types of light rays and/or light of varyingwavelengths. For example, the light sensor 866 can detect LiDAR,infrared, and/or red lights. The light sensor 866 can also detectchanges in light between ambient lighting (e.g., lighting from a window)and LED lighting or other fixture lighting.

The sound sensor 868 can detect changes in decibels, vibrations, slightsounds, or other anomalous sounds in an area proximate to the locationof the sensor device 108A. The sound sensor 868 may not monitoroccupants' conversations. However, the sound sensor 868 can beconfigured to detect the occupant's voice when the occupant is speakingto the sensor device 108A (e.g., refer to FIGS. 5A-B). For example, whenthe occupant asks the sensor device 108A for an update on conditions inthe building, the sound sensor 868 can detect the occupant's voice sothat the sensor device 108A can respond accordingly.

The motion sensor 872 can detect movement in an area proximate to thelocation of the sensor device 108A. For example, the motion sensor 872can detect slight movements (e.g., a curtain moving in a breeze from anopen window) as well as rapid, fast, or otherwise unexpected movements(e.g., a child or pet running through a room). Sometimes, the motionsensor 872 can be the same as the user presence sensor 860. Sometimes,the sensors 872 and 860 can be separate, as shown in FIG. 8A.

The image sensor 874 can capture image data of an area proximate to thelocation of the sensor device 108A. The image sensor 874 may notactively monitor activity in a room. Instead, the image sensor 874 canbe triggered, by the sensor controller 852, to activate and captureimage data of the room. For example, as depicted in FIGS. 5A-B, when thesensor device 108C detects the occupant 502A asking for a status updateon the building 500, the sensor device 108C can ping other sensordevices, such as the sensor device 108A, and request one or more typesof sensor signals. The one or more types of sensor signals can includeimage data. Thus, upon receiving the request, the sensor device 108A cancapture image data of the room 504B. Otherwise, the sensor device 108Amay not continuously capture image data of the room 504B. This isbeneficial to ensure and protect occupant privacy. The image data caninclude video feeds and/or still images.

The communication interface 862 can facilitate communication (e.g.,wired or wireless) with the other components, 830 and 840, comprisingthe sensor device 108A. The communication interface 862 can alsofacilitate communication between the sensor device 108A, the centralizedhub 102, other sensor devices 108A-N, sensors, and/or security system ofthe building.

The light signaling component 830 can include a light source 832, acontroller 834, a communication interface 836, and an optional powersource 838. The light source 832 can be any form of lighting, includingbut not limited to an LED light strip (e.g., refer to FIGS. 7 and 8B).The light source 832 can emit different colors, patterns, symbols basedon guidance or other signaling instructions communicated to the lightsignaling component 830 by the sensor controller 852. The controller 834can be configured to activate the light source 832 based on receiving anactivation signal/instruction from the sensor controller 852. Thecommunication interface 836 can allow the light signaling component 830to communicate with the sensor controller 852 and/or one or more othercomponents of the sensor device 108A. As mentioned, the power source 838can power the light signaling component 830. Sometimes, the lightsignaling component 830 may not include the power source 838 and caninstead rely on power from the external power supply 870 that providespower to the sensor device 108A as a whole.

The audio signaling component 840 can include a speaker 842, acontroller 844, a communication interface 846, stored audio signals 848,and an optional power source 850. The speaker 842 can be any form ormechanism to output audio cues/instructions (e.g., refer to FIGS. 7 and8B). The speaker 842 can emit audio/verbal instructions to an occupantin the building based on signaling instructions communicated to theaudio signaling component 840 by the sensor controller 852. Thecontroller 844 can be configured to activate the speaker 842 based onreceiving an activation signal/instruction from the sensor controller852. The communication interface 846 can allow the audio signalingcomponent 840 to communicate with the sensor controller 852 and/or anyother components of the sensor device 108A.

The stored audio signals 848 can include a plurality of verbalinstructions that are associated with each possible egress escape planout of a room that the sensor device 108A is located in. Therefore, whenthe sensor controller 852 transmits an activation signal to the audiosignaling component 840, the activation signal can indicate which of thestored audio signals from the stored audio signals 848 should be played.Then, the controller 844 can activate the speaker 842 by having thespeaker output the selected audio signals from the stored audio signals848. As mentioned, the power source 850 can power the audio signalingcomponent 840. Sometimes, the audio signaling component 840 may notinclude the power source 850 and can instead rely on power from theexternal power supply 870 that provides power to the sensor device 108Aas a whole.

FIG. 8B depicts an example system for providing emergency guidance andadvisement. In this example room 800, a door 802 is fitted with a firstLED strip 812. The first LED strip 812 can be attached on top of amolding of the door 802 or anywhere else along a perimeter of the door802. A window 804 is also fitted with a second LED strip 810, which canbe attached on top of a molding of the window 804 or anywhere else alonga perimeter of the window 804. This way, the first and second LED strips812 and 810 are not visible to an occupant or at least are notprominently displayed in the room 800.

In this example, sensor device 108A is also configured to a wall of theroom 800. The sensor device 108A can be retrofitted into an existingsocket in the wall. As described throughout this disclosure (e.g., referto FIG. 6 ), the sensor device 108A can be a plug-in that is inputtedinto an outlet in the room 800 or otherwise integrated into any otherexisting features in the room 800. Here, the sensor device 108A supportsaudio output. Thus, the sensor device 108A communicates with the firstand second LED strips 812 and 810 to display additional and/oralternative signals to an occupant during an emergency or securityevent. The sensor device 108A can also communicate with a mobile deviceof the occupant (not depicted) to provide guidance or other instructionsduring the emergency or security event. The strips 812 and 810 and thesensor device 108A can communicate through a wired and/or wirelessconnection, as previously discussed throughout this disclosure, whereina communication signal (e.g., activation signal) between the sensordevice 108A and the first LED strip 812 is signal 820B and acommunication signal between the sensor device 108A and the second LEDstrip 810 is signal 820A. During an emergency or security event and oncethe sensor device 108A selects an optimal egress escape plan, the sensordevice 108A can communicate visual signaling instructions to the firstand/or second LED strips 812 and 810 via the signals 820B and 820A,respectively.

For example, if the selected egress escape plan requires the occupant toexit through the door 802, the sensor device 108A can prompt (e.g., sendan activating signal to) the first LED strip 812 to turn green, depictarrows, and/or flash. The sensor device 108A can also prompt the secondLED strip 810 to turn red and/or depict “X” signals so that the occupantunderstands not to exit through the window 804. The sensor device 108Acan optionally, additionally, or alternatively output audio messagesinstructing the occupant about how to exit the room 800.

Moreover, the sensor device 108A can communicate with lights 814 in theroom 800 via signal 820C. Sometimes, the room 800 may not include thefirst and second LED strips 812 and 810. Instead, the sensor device 108Acan communicate with existing features in the room 800 to provideguidance to occupants located therein. For example, the sensor device108A can transmit light signaling instructions to the light 814 via thesignal 820C. The instructions can cause the light 814 to turn on, off,flash, etc. The light 814 can therefore assist in guiding the occupantto safely exit the room 800. The sensor device 108A can also communicatewith one or more other existing features in the room 800, including butnot limited to TVs, display screens, thermostats, alert/securitysystems, and smart appliances (e.g., stove, microwave) to provideguidance, instructions, or other notifications to the occupants locatedtherein.

FIG. 9 is an example apparatus 900 for providing emergency and/orsecurity guidance and advisement in accordance with the presentdisclosure. In this example, the apparatus 900 is configured as anelectrical power outlet that includes one or more receptacles 901. Theapparatus 900 is configured to include a sensor device 902 (e.g., thesensor devices 108A-N), an emergency and security event detector 904, acommunication device 906, a speaker 908, and a display device 910. Inother examples, the apparatus 900 can include one or more othercomponents described throughout this disclosure. For example, theapparatus 900 can include only the sensor device 906, which can includeone or more of the components described in reference to the sensordevices 108A-N throughout this disclosure (e.g., refer to FIGS. 2A-C).The apparatus 900 can also include components such as the emergencyegress module 206 and/or the security module 208 depicted and describedin reference to FIG. 2B. The apparatus 900 can further include one ormore of the sensors described throughout this disclosure, such as thesensors depicted and described in reference to FIG. 8A. Variousconfigurations of the apparatus 900 are possible.

The sensor device 902 can include a suite of sensors, as describedthroughout this disclosure, that can be configured to detect differenttypes of signals in an area proximate to the location of the apparatus900. Sensed signals can be recorded locally in the apparatus 900 and/orin one or more remote computing devices, such as the centralized hub 102(e.g., refer to FIGS. 1-2 ). As described herein, the user detector 602can be of various types, such as motion sensors and cameras.

The emergency and security event detector 904 can be configured todetect information about an emergency or security event. The emergencyand security event detector 904 can be part of the sensor device 902 orcan be separate. For example, signals can be detected by the sensordevice 902 and transmitted to the emergency and security event detector904. The emergency and security event detector 904 can then analyze thesignals and determine whether the signals indicate that an emergency orsecurity event is occurring in the building.

The communication device 906 is included in the apparatus 900 andconfigured to enable data communication with the centralized hub 102and/or other computing devices, sensor devices, and/or sensorspositioned throughout the building. The communication device 906 caninclude a wireless and/or wired data communication interface.

The speaker 908 operates to generate sounds, such as audible cues,horns, or verbal messages for egress guidance and instructions. Thespeaker 908 can also provide updates to the occupants about currentconditions in the building. For example, when an occupant asks theapparatus 900 for an update on what is happening in the building, statusinformation can be outputted by the speaker 908. The speaker 908 canalso be used to supplement other fixed audio devices or act as asubstitute if fixed audio devices are not functioning in the building.Such sounds can complement visual signs in emergencies where smokeintensity can diminish or preclude the ability to see the visual signs.

The display device 910 operates to display visual signs that can guidean occupant along an exit route. In some examples, the display device910 includes a display screen that is provided in the apparatus 900 anddisplays information with visual signs thereon. In addition oralternatively, the display device 910 operates as a projector thatprojects a lighted sign on another object, such as a wall, a floor, or aceiling. In the illustrated example, the display device 910 projects alighted arrow on the floor to guide the occupant in a direction to exit.

FIG. 10 is another example apparatus 930 for providing emergency and/orsecurity guidance and advisement in accordance with this presentdisclosure. The apparatus 930 is configured similar to the apparatus 900except that the apparatus 930 is implemented as an electrical switchhaving a switch button 932. Similarly to the apparatus 900, theapparatus 930 can include at least one of the sensor device 902, theemergency and security event detector 904, the communication device 906,the speaker 908, and the display device 910. As the apparatus 930 issimilar to the apparatus 900, the description of the apparatus 900 isincorporated by reference with respect to the apparatus 930.

FIG. 11 is a conceptual diagram depicting a building 1100 with sensordevices positioned throughout that provide guidance and prompts tobuilding occupants during an identified emergency. The guidance andprompts described in reference to FIG. 11 can be determined by thecentralized hub 102. More specifically, the emergency egress module 206of the centralized hub 102 can determine appropriate guidance and/orprompts for the building occupants based on detected conditions in thebuilding 1100. The guidance and/or prompts described in reference toFIG. 11 can also be determined by the emergency egress module 206 andapplied to security events such as break-ins and burglaries, as detectedand determined by the security module 208. For illustrative purposes,the guidance and prompts determined by the emergency egress module 206are described in relation to a fire emergency in the building.

The building 1100 can be a house that includes a lower level 1102 and astairway 1104 that goes to an upper level. The upper level includes ahallway 1105, a first bedroom 1106, and a second bedroom 1108. Acentralized hub 1110 (e.g., the centralized hub 102 described inreference to FIGS. 1-2 ) has determined a floor map 1112 that mapspathways within the building 1100. The floor map 1112 may have beendetermined from motion detection information obtained by detectingmotion of occupants as they moved throughout the building 1100. Thefloor map 1112 can also be determined as described in reference to theemergency egress module 206 in FIG. 2B. Sensor devices 1114, 1116, 1118,1120, 1124, and 1126 can also be placed in each of the rooms orlocations 1102, 1105, 1106, and 1108 in the building 1100. The sensordevices 1114, 1116, 1118, 1120, 1124, and 1126 can be connected to thecentralized hub 1110, through one or more wired and/or wirelessconnections.

The building 200 can include various additional sensor devices, such assmart thermostat devices, that can detect different types of signals inthe building 200 and communicate other information to the centralizedhub 102, as described throughout this disclosure (e.g., refer to FIGS.1-2 ).

In general, devices that communicate with the centralized hub 1110 caninclude one or more of a smart thermostat, smoke detector, smart outletcovers, sensors, and sensor devices located, e.g., on doors and windowsor in other locations in the building 200. A given device may provideone function or multiple functions (e.g., a smart outlet cover mayinclude a motion detector as well as one or more additional sensors).

Although one centralized hub 1110 is shown, multiple monitoring devicesmay be included in the building 1100, such as one monitoring device perroom. Each of the monitoring devices can operate similarly to thecentralized hub 1110. For example, each of the monitoring devices cantake turns performing functions of the centralized hub 1110 (e.g., referto the centralized hub 102 in FIG. 2B). As another example, each of thesensor devices 1114, 1116, 1118, 1120, 1124, and 1126 can take turnsperforming functions of the centralized hub 1110. Moreover, a smartthermostat, existing alert system, smoke detector, or other devices inthe building 1100 can also be secondary monitoring systems. A secondarymonitoring system (and the centralized hub 1110) can include varioussensors (e.g., for audio, light, temperature, fire, smoke, motiondetection, etc.) and/or can communicate with other sensors included inan area monitored by the respective monitoring device.

In some implementations, the centralized hub 1110 can be a mastermonitoring system and other monitoring devices, such as the sensordevices 1114, 1116, 1118, 1120, 1124, and 1126 can be secondarymonitoring systems. In some implementations, each secondary monitoringsystem can take over control as a new master monitoring system if thecentralized hub 1110 is out of commission (e.g., consumed by fire), asdescribed above. A new master monitoring system can operate usinglast-received information from the centralized hub 1110 and informationreceived from other secondary monitoring systems. In someimplementations, all monitoring systems located in the building 1100 canact as peer devices (e.g., pre-emergency and/or during an emergency),with no device designated as a master monitoring device.

As mentioned, devices included in the building 1100 can connect to thecentralized hub 1110 using one or more wired or wireless connections.Additionally or alternatively, devices in the building 1100 can connectto a cloud based service, to upload information and download informationprovided by other devices, so that a given device can send and receivedata even if in a home network is compromised, e.g., by some type ofemergency. For example, during a disaster, devices may not be able tocommunicate on a local network, but a smart thermostat in one room andthe centralized hub 1110 may each be able to communicate with the cloudservice (e.g., using a cellular network) and thereby exchangeinformation with each other, using the cloud service as an intermediary.

Various devices, e.g., secondary monitoring systems or other devicesthat include motion detection, such as the sensor devices 1114, 1116,1118, 1120, 1124, and 1126, can provide motion detection information tothe centralized hub 1110 and/or to secondary monitoring system(s). Eachmotion detection device can have a known location within the building1100, and can provide a device identifier along with provided motiondetection information. The centralized hub 1110 can use the receivedmotion detection information to determine the floor map 1112 of thebuilding 1100. The floor map 1112 indicates paths into and out of thebuilding 1100, and paths into and out of respective rooms or on thestairway 1104.

The motion detection information can indicate the paths that occupantsfrequently use while moving within the building 1100. Paths can beidentified by time and location of detected motion, as well as directionof motion as indicated by successive motion detection data points. Forexample, first, second and third motion sensors may detect motion atfirst, second, and third time points that are one second apart,indicating that a user moved between locations associated with thefirst, second, and third motion sensors. Frequency of movement, overtime, can indicate main paths throughout the building 1100. For example,motion detectors may detect occasional movement of an occupant in acorner of a room (e.g., by a dresser), but may more often detectmovement of occupants in hallways, through doorways, on stairs, etc. Thecentralized hub 1110 may know which sensors are in proximity to doors(e.g., room doors, exit doors) and windows, and can identify paths thatlead into and out of rooms and out of the building 1100.

Sensors can also be located on doors and/or windows. The centralized hub1110 can determine an exit path by detecting movement of an occupanttowards a door and then opening of that door. As a similar example, thecentralized hub 1110 can detect a path that includes an exit bydetecting the opening of a door when an occupant enters, and thendetecting continuous movement of the occupant through the building 1100to a location within the building. The centralized hub 1110 can identifypath segments within the building 1100 that interconnect and that leadto an exit (e.g., door, window).

During runtime, the centralized hub 1110 (or a secondary monitoringsystem) can receive information that indicates a presence of anemergency within the building 1100. For example, the centralized hub1110 can receive (or can generate) information that indicates thepresence of a fire 1128 on the lower level 1102 of the building 1100.The presence of the fire 1128 can be determined, for example, based onone or more received temperature readings being more than a thresholdtemperature. As another example, the centralized hub 1110 can receive afire indication signal from one or more smoke detection devices. Otherfire detection approaches can include IR (Infra-Red) fire detection andrate of rise temperature detection. Fire indication information canindicate which location(s) in the building 1100 are on fire (orsufficiently close to a fire so as to be avoided by occupants of thebuilding 1100).

The centralized hub 1110 can determine, based on the floor map 1112 andthe received fire indication information, one or more exit routes thatcan be used by occupants to exit the building 1100. The exit routes caninclude portions of the floor map 1112 that avoid the locations withinthe building 1100 that have been identified as locations to be avoided.For example, based on the location of the fire 1128 being on the leftside of the building 1100, the centralized hub 1110 can determine thatthe stairs 1104 are currently usable. Accordingly, the centralized hub1110 can determine an exit path that routes upstairs occupants down thestairs 1104 and out a front door 1130.

After determining the exit route(s), the centralized hub 1110 cangenerate and send signaling instructions to various sensor deviceslocated in the building 1100, for the sensor devices to emit signal(s)to guide the occupant to an exit route that will safely lead theoccupant out of the building 1100. For example, the centralized hub 1110can send audio and/or visual signaling instructions to one or more ofthe sensor devices 1114, 1116, 1118, 1120, 1124, and 1126.

Sensor devices 1114, 1116, 1118, 1120, 1124, and 1126 can emitmulti-colored, strobing, LED (Light Emitting Diode) laser light, and canbe mounted low, at exit points (e.g., door, window) in each room. LEDguiding lights, can be mounted low in outlet-type components, inpathways leading to egresses from the building 1100. Sensor devices1114, 1116, 1118, 1120, 1124, and 1126 can emit various audio and visualcues to the occupant, for example. For instance, sensor devices 1114,1116, 1118, 1120, 1124, and 1126 can include flashing lights that mayindicate a direction an occupant is to take to proceed to (or stay one)an exit route. A series of flashing lights (e.g., in a hallway) mayindicate a presence and direction of an exit route. Sensor devices 1114,1116, 1118, 1120, 1124, and 1126 can also be placed onto doors and/orwindows, to indicate presence of respective doors and windows, and toindicate whether a given door or window is part of an exit route.Different colors can indicate inclusion or exclusion of a given door,window, or pathway on an exit route. For example, a flashing red signal(e.g., a red “X”) on a doorway may indicate that the doorway is to beavoided (and the door kept shut). A flashing green light may indicatethat the given door, window, or path segment is part of the exit route.

Sensor devices 1114, 1116, 1118, 1120, 1124, and 1126 can be configuredto play audio instructions for an occupant, for providing directionalguidance towards egresses, as described herein. Audio instructions caninclude a fire status description (e.g., “a fire has been detecteddownstairs”), directional clues (e.g., “go out of the door and to yourleft”), or more detailed instructions (e.g., “place a wet towel underthe door and leave the door closed”). Audio instructions can be specificto the particular room where the occupant is located, based on thelocation of the room, the location of the detected fire, a determinedexit route, and potential detected levels of concern, panic, oruncertainty of the occupant (e.g., refer to the centralized hub 102 inFIG. 2B).

For the particular example of the fire 1128 located in the left of thelower level 1102, the centralized hub 1110 can emit a lighteddirectional signal and an audio instruction 1132 directing occupantslocated in the lower level 1102 to proceed to and exit the front door1130. Signaling instructions can be sent from the centralized hub 1110to the sensor device 1118 located near an entry to the lower level 1102,for the sensor device 1118 to play an audio instruction 1136 directingthe occupant to not enter the lower level 1102.

Signaling instructions can also be sent from the centralized hub 1110 tosensor devices 1124 and 1126 located in the room 1106, for the sensordevices 1124 and 1126 to direct occupants located in the room 1106 outof the building 1100. For example, the sensor devices 1124 and 1126 canemit lighted arrows that direct the occupant to a bedroom door 1136 andout of the room 1106. Sensor devices located on or near the bedroom door1136 (e.g., a sensor device 1137) can emit, in response to receivedinstructions, signals (e.g., lighted) indicating the presence of thebedroom door 1136 and that the occupant is to go through the bedroomdoor 1136. Sensor devices located on or near windows 1138 and 1140 canemit, in response to received instructions, signals (e.g., lighted)indicating that the windows 1138 and 1140 are not part of a recommendedexit route. The sensor device 1126 (or another device) can, in responseto a received instruction, emit an audio instruction 1142 that directsoccupants in the room 1106 to exit the room 1106 and proceed to thestairs 1103. The device 1122, located in the hallway 1105, can emit alighted arrow directing users down the hallway 1105 and audioinstructions 1144 that directs the occupants to the stairs 1103. Thedevice 1114, also located in the hallway 1105, can emit audioinstructions 1134 that directs the occupants down the stairs 1103 and toexit through the front door 1130.

Signaling instructions similar to those sent to sensor devices 1124 and1126 in the room 1106 can also be sent to sensor devices 1120 and 1146in the room 1108. Signals emitted by sensor devices 1120 and 1146 in theroom 1108, including an audio instruction 1146 played by the sensordevice 1120, can direct occupants out of the room 1108 (e.g., through adoor 1147 and down the stairs 1103), rather than out a window 1148 or awindow 1150.

Other types of signaling instructions and corresponding signals can begenerated in the building 1100. For example, information can be sent tomobile devices of occupants of the building 1100 that direct theoccupants to and on the determined exit routes. The centralized hub1110, secondary monitoring systems, and/or an application running on amobile device may know where the mobile device (and associated occupant)are located within the building 1100, with respect to the fire and thedetermined exit routes. Such knowledge can be used to tailorinstructions that are sent to and displayed (or played) on a givenmobile device.

Other devices in the building may receive and present informationrelated to the fire 1128 and recommended evacuation of the building1100. For example, the centralized hub 1110 can communicate with variouscomputing devices or displays located within the building 1100. Forexample, the centralized hub 1110 can send information or signalinginstructions to one or more desktop computing devices, smarttelevisions, or other devices located within the building 1100. Thecomputing devices can be configured to display information (e.g., a firewarning, exit route information), based on information received from thecentralized hub 1110. In some implementations, the centralized hub 1110can remotely control (e.g., turn on) devices that include a display, andinstruct the devices to display (and/or play) information useful forevacuation of the building 1100, such as exit route information that isspecific to the location of the fire 1128 and the location of therespective device (e.g., a smart television in the lower level 1102 maydisplay different information from a smart television in the room 1108).

FIG. 12 shows an example of a computing device 1200 and an example of amobile computing device that can be used to implement the techniquesdescribed here. The computing device 1200 is intended to representvarious forms of digital computers, such as laptops, desktops,workstations, personal digital assistants, servers, blade servers,mainframes, and other appropriate computers. The mobile computing deviceis intended to represent various forms of mobile devices, such aspersonal digital assistants, cellular telephones, smart-phones, andother similar computing devices. The components shown here, theirconnections and relationships, and their functions, are meant to beexemplary only, and are not meant to limit implementations of theinventions described and/or claimed in this document.

The computing device 1200 includes a processor 1202, a memory 1204, astorage device 1206, a high-speed interface 1208 connecting to thememory 1204 and multiple high-speed expansion ports 1210, and alow-speed interface 1212 connecting to a low-speed expansion port 1214and the storage device 1206. Each of the processor 1202, the memory1204, the storage device 1206, the high-speed interface 1208, thehigh-speed expansion ports 1210, and the low-speed interface 1212, areinterconnected using various busses, and can be mounted on a commonmotherboard or in other manners as appropriate. The processor 1202 canprocess instructions for execution within the computing device 1200,including instructions stored in the memory 1204 or on the storagedevice 1206 to display graphical information for a GUI on an externalinput/output device, such as a display 1216 coupled to the high-speedinterface 1208. In other implementations, multiple processors and/ormultiple buses can be used, as appropriate, along with multiple memoriesand types of memory. Also, multiple computing devices can be connected,with each device providing portions of the necessary operations (e.g.,as a server bank, a group of blade servers, or a multi-processorsystem).

The memory 1204 stores information within the computing device 1200. Insome implementations, the memory 1204 is a volatile memory unit orunits. In some implementations, the memory 1204 is a non-volatile memoryunit or units. The memory 1204 can also be another form ofcomputer-readable medium, such as a magnetic or optical disk.

The storage device 1206 is capable of providing mass storage for thecomputing device 1200. In some implementations, the storage device 1206can be or contain a computer-readable medium, such as a floppy diskdevice, a hard disk device, an optical disk device, or a tape device, aflash memory or other similar solid state memory device, or an array ofdevices, including devices in a storage area network or otherconfigurations. A computer program product can be tangibly embodied inan information carrier. The computer program product can also containinstructions that, when executed, perform one or more methods, such asthose described above. The computer program product can also be tangiblyembodied in a computer- or machine-readable medium, such as the memory1204, the storage device 1206, or memory on the processor 1202.

The high-speed interface 1208 manages bandwidth-intensive operations forthe computing device 1200, while the low-speed interface 1212 manageslower bandwidth-intensive operations. Such allocation of functions isexemplary only. In some implementations, the high-speed interface 1208is coupled to the memory 1204, the display 1216 (e.g., through agraphics processor or accelerator), and to the high-speed expansionports 1210, which can accept various expansion cards (not shown). In theimplementation, the low-speed interface 1212 is coupled to the storagedevice 1206 and the low-speed expansion port 1214. The low-speedexpansion port 1214, which can include various communication ports(e.g., USB, Bluetooth, Ethernet, wireless Ethernet) can be coupled toone or more input/output devices, such as a keyboard, a pointing device,a scanner, or a networking device such as a switch or router, e.g.,through a network adapter.

The computing device 1200 can be implemented in a number of differentforms, as shown in the figure. For example, it can be implemented as astandard server 1220, or multiple times in a group of such servers. Inaddition, it can be implemented in a personal computer such as a laptopcomputer 1222. It can also be implemented as part of a rack serversystem 1224. Alternatively, components from the computing device 1200can be combined with other components in a mobile device (not shown),such as a mobile computing device 1250. Each of such devices can containone or more of the computing device 1200 and the mobile computing device1250, and an entire system can be made up of multiple computing devicescommunicating with each other.

The mobile computing device 1250 includes a processor 1252, a memory1264, an input/output device such as a display 1254, a communicationinterface 1266, and a transceiver 1268, among other components. Themobile computing device 1250 can also be provided with a storage device,such as a micro-drive or other device, to provide additional storage.Each of the processor 1252, the memory 1264, the display 1254, thecommunication interface 1266, and the transceiver 1268, areinterconnected using various buses, and several of the components can bemounted on a common motherboard or in other manners as appropriate.

The processor 1252 can execute instructions within the mobile computingdevice 1250, including instructions stored in the memory 1264. Theprocessor 1252 can be implemented as a chipset of chips that includeseparate and multiple analog and digital processors. The processor 1252can provide, for example, for coordination of the other components ofthe mobile computing device 1250, such as control of user interfaces,applications run by the mobile computing device 1250, and wirelesscommunication by the mobile computing device 1250.

The processor 1252 can communicate with a user through a controlinterface 1258 and a display interface 1256 coupled to the display 1254.The display 1254 can be, for example, a TFT (Thin-Film-Transistor LiquidCrystal Display) display or an OLED (Organic Light Emitting Diode)display, or other appropriate display technology. The display interface1256 can comprise appropriate circuitry for driving the display 1254 topresent graphical and other information to a user. The control interface1258 can receive commands from a user and convert them for submission tothe processor 1252. In addition, an external interface 1262 can providecommunication with the processor 1252, so as to enable near areacommunication of the mobile computing device 1250 with other devices.The external interface 1262 can provide, for example, for wiredcommunication in some implementations, or for wireless communication inother implementations, and multiple interfaces can also be used.

The memory 1264 stores information within the mobile computing device1250. The memory 1264 can be implemented as one or more of acomputer-readable medium or media, a volatile memory unit or units, or anon-volatile memory unit or units. An expansion memory 1274 can also beprovided and connected to the mobile computing device 1250 through anexpansion interface 1272, which can include, for example, a SIMM (SingleIn Line Memory Module) card interface. The expansion memory 1274 canprovide extra storage space for the mobile computing device 1250, or canalso store applications or other information for the mobile computingdevice 1250. Specifically, the expansion memory 1274 can includeinstructions to carry out or supplement the processes described above,and can include secure information also. Thus, for example, theexpansion memory 1274 can be provide as a security module for the mobilecomputing device 1250, and can be programmed with instructions thatpermit secure use of the mobile computing device 1250. In addition,secure applications can be provided via the SIMM cards, along withadditional information, such as placing identifying information on theSIMM card in a non-hackable manner.

The memory can include, for example, flash memory and/or NVRAM memory(non-volatile random access memory), as discussed below. In someimplementations, a computer program product is tangibly embodied in aninformation carrier. The computer program product contains instructionsthat, when executed, perform one or more methods, such as thosedescribed above. The computer program product can be a computer- ormachine-readable medium, such as the memory 1264, the expansion memory1274, or memory on the processor 1252. In some implementations, thecomputer program product can be received in a propagated signal, forexample, over the transceiver 1268 or the external interface 1262.

The mobile computing device 1250 can communicate wirelessly through thecommunication interface 1266, which can include digital signalprocessing circuitry where necessary. The communication interface 1266can provide for communications under various modes or protocols, such asGSM voice calls (Global System for Mobile communications), SMS (ShortMessage Service), EMS (Enhanced Messaging Service), or MMS messaging(Multimedia Messaging Service), CDMA (code division multiple access),TDMA (time division multiple access), PDC (Personal Digital Cellular),WCDMA (Wideband Code Division Multiple Access), CDMA2000, or GPRS(General Packet Radio Service), among others. Such communication canoccur, for example, through the transceiver 1268 using aradio-frequency. In addition, short-range communication can occur, suchas using a Bluetooth, WiFi, or other such transceiver (not shown). Inaddition, a GPS (Global Positioning System) receiver module 1270 canprovide additional navigation- and location-related wireless data to themobile computing device 1250, which can be used as appropriate byapplications running on the mobile computing device 1250.

The mobile computing device 1250 can also communicate audibly using anaudio codec 1260, which can receive spoken information from a user andconvert it to usable digital information. The audio codec 1260 canlikewise generate audible sound for a user, such as through a speaker,e.g., in a handset of the mobile computing device 1250. Such sound caninclude sound from voice telephone calls, can include recorded sound(e.g., voice messages, music files, etc.) and can also include soundgenerated by applications operating on the mobile computing device 1250.

The mobile computing device 1250 can be implemented in a number ofdifferent forms, as shown in the figure. For example, it can beimplemented as a cellular telephone 1280. It can also be implemented aspart of a smart-phone 1282, personal digital assistant, or other similarmobile device.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichcan be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms machine-readable medium andcomputer-readable medium refer to any computer program product,apparatus and/or device (e.g., magnetic discs, optical disks, memory,Programmable Logic Devices (PLDs)) used to provide machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as a machine-readable signal.The term machine-readable signal refers to any signal used to providemachine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device(e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor)for displaying information to the user and a keyboard and a pointingdevice (e.g., a mouse or a trackball) by which the user can provideinput to the computer. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback (e.g., visual feedback,auditory feedback, or tactile feedback); and input from the user can bereceived in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here), or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (LAN), a wide area network (WAN), and the Internet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of thedisclosed technology or of what may be claimed, but rather asdescriptions of features that may be specific to particular embodimentsof particular disclosed technologies. Certain features that aredescribed in this specification in the context of separate embodimentscan also be implemented in combination in a single embodiment in part orin whole. Conversely, various features that are described in the contextof a single embodiment can also be implemented in multiple embodimentsseparately or in any suitable subcombination. Moreover, althoughfeatures may be described herein as acting in certain combinationsand/or initially claimed as such, one or more features from a claimedcombination can in some cases be excised from the combination, and theclaimed combination may be directed to a subcombination or variation ofa subcombination. Similarly, while operations may be described in aparticular order, this should not be understood as requiring that suchoperations be performed in the particular order or in sequential order,or that all operations be performed, to achieve desirable results.Particular embodiments of the subject matter have been described. Otherembodiments are within the scope of the following claims.

1. A system for providing distributed security event monitoring in apremises, the system comprising: a central monitoring system fordetecting a security event in a premises; and a plurality of sensordevices positioned throughout the premises and configured to (i)generate signals based on passively detecting conditions on the premisesand (ii) emit signals that indicate guidance for people in the premiseswhen the security event is detected by the central monitoring system,wherein each of the plurality of sensor devices include a suite ofsensors, wherein each of the plurality of sensor devices is configuredto: generate the signals based on passively detecting, using the suiteof sensors, the conditions on the premises; and transmit the signalsdetected conditions to the central monitoring system; wherein thecentral monitoring system is configured to: receive the signals from oneor more of the plurality of sensor devices; correlate the signals togenerate data representing a combination of the detected conditions;determine whether the data representing the combination of the detectedconditions exceeds expected threshold conditions for the premises beyonda threshold amount; identify, based on determining that the datarepresenting the combination of the collective of detected conditionsexceeds the expected threshold conditions beyond the threshold amount, asecurity event on the premises; classify, based on providing the datarepresenting the combination of the detected conditions as input to amachine learning model, the security event, wherein the machine learningmodel was trained to identify a type of the security event usingtraining data that correlates information about conditions detected onpremises with different types of security events; generate, based on theclassified security event, instructions to produce audio or visualoutput at the plurality of sensor devices, wherein the output notifiesthe people on the premises about the security event; and transmit theinstructions to one or more of the plurality of sensor devices for theone or more of the plurality of sensor devices to emit signalsindicating information about the security event.
 2. The system of claim1, wherein the type of the security event includes at least one of aburglary, a theft, a break-in, a fire, a gas leak, and a flood, whereinclassifying, based on providing the data representing the combination ofthe detected conditions as input to a machine learning model, thesecurity event comprises: determining, based on (i) the type of thesecurity event and (ii) a magnitude in deviation of the combination ofthe detected conditions from the expected threshold conditions, aseverity level of the security event; and determining a location of thesecurity event based on (i) a map of the premises that was generated bythe central monitoring system, (ii) timestamps indicating when theconditions were detected by the plurality of sensor devices, and (iii)positioning information of each of the plurality of sensor devices onthe premises.
 3. The system of claim 2, wherein the central monitoringsystem is further configured to transmit, based on determining that theseverity level of the security event exceeds a threshold reportinglevel, a notification about the security event to emergency responsepersonnel.
 4. The system of claim 1, wherein the expected thresholdconditions are normal conditions on the premises that have beenidentified by the central monitoring system based on a historic spreadof conditions detected by the plurality of sensor devices over apredetermined period of time.
 5. The system of claim 1, wherein thecentral monitoring system is configured to generate instructions toproduce (i) audio output when the data representing the combination ofthe detected conditions satisfies a first output threshold condition and(ii) visual output when the data representing the combination of thedetected conditions satisfies a second output threshold condition,wherein the first output threshold condition includes detection of atleast one of a fire, smoke, and an obstruction of visual output devicesof the plurality of sensor devices, and wherein the second outputthreshold condition includes detection of at least one of a fire, abreak-in, a burglary, and an obstruction of audio output devices of theplurality of sensor devices.
 6. The system of claim 1, wherein thecentral monitoring system is further configured to: determine, beforedetection of a security event, a map of the premises, the map indicatingroutes within the premises, including exits out of the premises;determine, based on the map and the data representing the combinationthe detected conditions, one or more exit routes that can be used bypeople to exit the premises, wherein the data representing thecombination of the detected conditions include location informationindicating one or more locations on the premises where the securityevent may be located, and wherein the exit routes avoid the one or morelocations where the security event may be located; and transmitsignaling instructions to one or more of the plurality of sensor devicesfor the one or more of the plurality of sensor devices to emit signalsthat indicate to the people an exit route to exit the premises.
 7. Thesystem of claim 1, wherein the emitted signals comprise voice commands.8. The system of claim 7, wherein the voice commands comprise at leastone of instructions to guide the people to exit the premises andinformation about the security event, wherein the information about thesecurity event includes the type of the security event, a location ofthe security event, and a severity level of the security event.
 9. Thesystem of claim 1, wherein the emitted signals comprise light signals,wherein the light signals include directional signals that direct thepeople to an exit route that avoids a location of the security event.10. The system of claim 1, wherein the suite of sensors includes atleast one of a light sensor, an audio sensor, a temperature sensor, amotion sensor, a user presence sensor, an image sensor, and a smokesensor.
 11. The system of claim 10, wherein the audio sensor isconfigured to detect changes in decibels in an area proximate to alocation of the audio sensor.
 12. The system of claim 1, wherein theplurality of sensor devices are integrated into at least one of outletcovers, light switches, alert systems, thermostats, and light fixturesthat are installed on the premises.
 13. The system of claim 1, whereinone of the plurality of sensor devices is configured to operate as thecentral monitoring system and the other sensor devices of the pluralityof sensor devices are configured to communicate amongst each other. 14.The system of claim 1, wherein one or more of the plurality of sensordevices are configured to identify the security event and transmitinformation about the identified security event to the centralmonitoring system, wherein the central monitoring system is configuredto: receive data representing the identified security event from one ormore of the plurality of sensor devices; classify the security eventbased on the received data; generate instructions to produce output bythe plurality of sensor devices about the classified security event; andtransmit the instructions to the plurality of sensor devices.
 15. Thesystem of claim 1, wherein one or more of the plurality of sensordevices are configured to: detect audio input from a person on thepremises, wherein the audio input includes a request for informationabout a current state of the premises; and transmit the audio input anda respective timestamp to the central monitoring system, wherein thecentral monitoring system is further configured to: receive the audioinput and the respective timestamp; transmit, based on the request forinformation in the audio input, requests to each of the plurality ofsensor devices for conditions detected at a similar time as thetimestamp; receive, from one or more of the plurality of sensor devices,signals representing the conditions detected at the similar time as thetimestamp; identify the current state of the premises based on comparingthe received signals representing the detected conditions to historicthreshold conditions for the premises at the similar time as thetimestamp; generate instructions for the one or more of the plurality ofsensor devices to provide audio output to the person indicating thecurrent state of the premises; and transmit, to the one or more of theplurality of sensor devices, the instructions to provide audio output tothe person.
 16. The system of claim 1, wherein the central monitoringsystem is further configured to transmit the instructions to provideoutput to one or more mobile devices of the people on the premises,wherein the instructions cause the one or more mobile devices to outputat least one of audio signals, text messages, and push notificationsabout the security event.
 17. The system of claim 1, wherein each of theplurality of sensor devices further include an audio signal generatorand a visual signal generator, wherein the visual signal generatorincludes a projector that projects a lighted sign on a surface, andwherein the surface is one or more of a wall, a floor, and a ceiling onthe premises.
 18. The system of claim 1, wherein the premises is atleast one of a building, a home, and an apartment.
 19. A method forproviding distributed security event monitoring in a premises, themethod comprising: receiving, by a computing system and from a pluralityof sensor devices, signals representing detected conditions on apremises, wherein the plurality of sensor devices include a suite ofsensors that generate the signals based on passively detecting theconditions on the premises, and wherein the plurality of sensor devicesare positioned throughout the premises; correlating, by the computingsystem, the signals to generate data representing a combination of thedetected conditions; determining, by the computing system, whether thedata representing the combination collective of the detected conditionsexceeds expected threshold conditions for the premises beyond athreshold amount; identifying, by the computing system and based ondetermining that the data representing the combination of the collectiveof detected conditions exceeds the expected threshold conditions beyondthe threshold amount, a security event on the premises; classifying, bythe computing system and based on providing the data representing thecombination of the detected conditions as input to a machine learningmodel, the security event, wherein the machine learning model wastrained to identify a type of the security event using training datathat correlates information about conditions detected on premises withdifferent types of security events; generating, by the computing systemand based on the classified security event, instructions to produceaudio or visual output at the plurality of sensor devices, wherein theoutput notifies the people on the premises about the security event; andtransmitting, by the computing system to one or more of the plurality ofsensor devices, the instructions for the one or more of the plurality ofsensor devices to emit signals indicating information about the securityevent.
 20. The method of claim 19, wherein the computing system is atleast one of remote from the premises, centralized at the premises, andone of the plurality of sensor devices.